Data availability for TR.Price52WeekHighDate Screens

We are performing screens on Equities on XNYS and XTSE to find 52 week highs and lows for the 2 exchanges. The code has been running very robustly for several years without problems in the evening about 1 hour after the market closes. About 6 months ago, the same filters began intermittently returning NULL for XTSE at that time, but continued to work for XNYS. We discovered that if we ran at 6AM, then both XNYS and XTSE would run, so we changed our scheduling so that the routine would run then also.

Now, we are getting almost regular failures from both XNYS and XTSE at both times. It is as if the data is not prepared yet.

Here is a typical call from our code:

instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,' " + data[1] + " '),BETWEEN(TR.Price52WeekHighDate," + dateEnd + "," + dateEnd + "))";

Where data[1] might be XNYS and dateEnd is say 20211124

Does anyone have any insight regarding this SCREEN, experienced similar degradation of service or knows anything about its timing/data availability?

I would be happy to share the complete code if someone would be willing to look at it.

Find more posts tagged with

eikon

screening

eikon-data-api

Accepted answers

jason.ramchandani01

@dgarrard I ran this at 10:49am London which is 05:49 Montreal (so slightly before your 6am). I ran the screener query in Python but it uses the same backend service as the older API you are using.

instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + "XNYS" + "'),BETWEEN(TR.Price52WeekHighDate," + "20211201" + "," + "20211201" + "))"

ek.get_data(instrument,'TR.PRICECLOSE')

I then ran it for TSE at 10:54 London (05:54 Montreal):

instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + "XTSE" + "'),BETWEEN(TR.Price52WeekHighDate," + "20211201" + "," + "20211201" + "))"

ek.get_data(instrument,'TR.PRICECLOSE')

Are you able to replicate this?

I checked with the Content team and there has been no change to the closing run times which are 07:00 AM (12:00 GMT) while TSE is at 02:00 New York time (07:00 GMT). However, the calculated field is scheduled to be updated within 4hrs of the market close. So this is testable. I hope this can help. Please let me know.

All comments

jason.ramchandani01

@dgarrard It is possible that the times of the closing runs may have changed or something like that might have happened - this will require further investigation. Please can you post the full API call code that is failing so we can try to replicate and investigate further. Also is it failing all the time or only on some days. The output of the screener is a list of instruments - so is the failure that the instruments you expect to be in that list are not there or some other failure?

Apols we need a few more details. thanks

dgarrard

Thank you @jason.ramchandani01

As you suggest, it appears that there might be some change in the window during which the data is available for TR.Price52WeekHighDate. It used to be available without much restriction over the past 9 - 10 years if we called anytime after 5PM, but these problems with the API returning the word "NULL" to the SCREEN request started earlier this year on Toronto, and now are happening with XNYS. The last week has seen no replies on both exchanges at about 5PM and 6AM almost every day now.

The code is below. We call it like this,

C:\\PNC\\Stats\\PNC_YearHiLo.exe Hi.TO 20211124 20211124

C:\\PNC\\Stats\\PNC_YearHiLo.exe Lo.TO 20211124 20211124

C:\\PNC\\Stats\\PNC_YearHiLo.exe Hi.N 20211124 20211124

C:\\PNC\\Stats\\PNC_YearHiLo.exe Lo.N 20211124 20211124

where, in the first example, Hi is mapped to return the highs and TO will be coded for Toronto. The 20211124 is the session date.

If it works I get a list of stocks and I prepend and append rows ~Beginning of Data and ~End of Data. If the call fails my EXE returns:

~Beginning of Data

NULL

~End of Data

Regards,

David

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using EikonDesktopDataAPI;
using Dex2;

namespace CSharpEikonnect
{
    public partial class Form1 : Form,_IEikonDesktopDataAPIEvents,_IRDataEvents
    {

        EikonDesktopDataAPI.EikonDesktopDataAPI MyEikonDesktopDataAPI = null;

        Dex2.Dex2Mgr MyDex2Mgr = null;
        Dex2.IDex2Mgr2 MyDex2MgrADC = null;
        int MyDex2Cookie = 0;
        Dex2.RData MyRData = null;
        private string argument = "";
        private string startDate = "";
        private string endDate = "";
        public Form1(string argument,string dateStart,string dateEnd)
        {
            //This call is required by the designer.
            InitializeComponent();
            this.argument = argument;
            this.startDate = dateStart;
            this.endDate = dateEnd;
            this.WindowState = FormWindowState.Minimized;
            // Add any initialization after the InitializeComponent() call.
            UpdateUserFormAccordingToConnectionStatus(EikonDesktopDataAPI.EEikonStatus.Disconnected);
            UpdateDex2FormAccordingToDex2Status(argument,dateStart,dateEnd);
        }

    //================================================================================================
    // Step 1 - Instantiate EikonDesktopDataAPILib.EikonDesktopDataAPI and connect to its outgoing interface =
    //================================================================================================

        private void ConnectToEikon()
        {
            //DisconnectFromEikon();
            //Create the Eikon Desktop object
            MyEikonDesktopDataAPI = new EikonDesktopDataAPI.EikonDesktopDataAPI();

            // Initialize the Eikon Desktop object to connect to Eikon
            if (MyEikonDesktopDataAPI != null)
            {

                EEikonDataAPIInitializeResult l_result = MyEikonDesktopDataAPI.Initialize();
                if (l_result != EEikonDataAPIInitializeResult.Succeed)
                {
                    //DisconnectFromEikon();
                    if (l_result == EEikonDataAPIInitializeResult.Error_Reinitialize)
                    {
                        DisconnectFromEikonBtn_Click(null, null);
                        DataRetrievedTxt.Text = "Reinitialization of EikonDesktopDataAPI is forbidden";
                    }
                    else if (l_result == EEikonDataAPIInitializeResult.Error_InitializeFail)
                    {
                        DisconnectFromEikonBtn_Click(null, null);
                        DataRetrievedTxt.Text = "Initialization of EikonDesktopDataAPI Fail";
                    }
                    UpdateUserFormAccordingToConnectionStatus(EEikonStatus.Disconnected);
                    return;
                }
            }
            MyEikonDesktopDataAPI.OnStatusChanged += OnStatusChanged;
        }


        private void DisconnectFromEikon()
        {
            // Release the Dex2 manager is necessary
            ReleaseDex2Mgr();

            // Release Eikonnect
            MyEikonDesktopDataAPI = null;
        }

        public void OnStatusChanged(EEikonStatus EStatus)
        {
            //Update the use form according to the status of the connection to Eikon
            UpdateUserFormAccordingToConnectionStatus(EStatus);
        }

        //======================================
        // Step 2 - Instantiate a Dex2 manager =
        //======================================


        private void CreateDex2Mgr()
        {
            // Create the Dex2 manager
            MyDex2Mgr = MyEikonDesktopDataAPI.CreateDex2Mgr();
            MyDex2MgrADC = (Dex2.IDex2Mgr2) MyDex2Mgr;

            // Initialize the Dex2 manager and retrieve a session cookie
            if (MyDex2Mgr != null)
                MyDex2Cookie = MyDex2MgrADC.Initialize(DEX2_MetadataCaller.DE_MC_ADC_POWERLINK);
        }

        private void ReleaseDex2Mgr()
        {
            //Release the RData if necessary
            MyRData = null;

            // Release the Dex2 manager
            MyDex2Mgr = null;
        }

        //======================================
        // Step #3 : Create a RData to request =
        //======================================


        private void CreateRData(string InstrumentIDList,
                                string FieldList,
                                string RequestParam,
                                string DisplayParam)
        {
            // Create a RData
            MyRData = MyDex2Mgr.CreateRData(MyDex2Cookie);
            MyRData.OnUpdate += OnUpdate;
            // Initialize the RData to request
            if (MyRData != null)
            {
                MyRData.InstrumentIDList = InstrumentIDList;
                MyRData.FieldList = FieldList;
                MyRData.RequestParam = RequestParam;
                MyRData.DisplayParam = DisplayParam;
                RequestDataBtn.Enabled = true;
            }
        }

        public void OnUpdate(DEX2_DataStatus DataStatus, object Error)
        {
            // Handle the Dex2 response for the RData requested
            UpdateUserFormAccordingToRDataResponse(DataStatus, Error);
        }

        //======================================================
        // Step #4 : Send the request to Dex2 and get the data =
        //======================================================

        private void RequestDex2Data()
        {
            //Send the RData request to Dex2
            if (MyRData != null)
                MyRData.Subscribe();
        }


        //==========================================================================================================================
        //==========================================================================================================================

        //----------------------------------------------------------------------


        private void UpdateDex2FormAccordingToDex2Status(string argument,string dateStart,string dateEnd)
        {
            if (MyDex2Mgr != null)
            {
                CreateDex2ManagerBtn.Enabled = false;
                ReleaseDex2ManagerBtn.Enabled = true;
                RDataGroupBox.Enabled = true;
                if (InstrumentsListTxt.TextLength != 0 && FieldsListTxt.TextLength != 0)
                    CreateRDataBtn.Enabled = true;
                else
                    CreateRDataBtn.Enabled = false;
            }
            else
            {
                if (!CreateDex2ManagerBtn.Enabled)
                {
                    ConnectToEikonBtn_Click(null, null);
                }
                else
                {
                    CreateDex2ManagerBtn_Click(null, null);
                    CreateRDataBtn_Click(this.argument,this.startDate,this.endDate,null);
                    RequestDataBtn_Click(null, null);
                }

            }

        }

        private void UpdateUserFormAccordingToConnectionStatus(EikonDesktopDataAPI.EEikonStatus EStatus)
        {
            switch (EStatus)
            {
                case EEikonStatus.Connected:
                    ConnectToEikonBtn.Enabled = false;
                    DisconnectFromEikonBtn.Enabled = true;
                    StatusPictureBox.Image = Properties.Resources.Green;
                    ToolStripStatusLabel1.Text = "Status : Connected to Eikon";
                    Dex2MgrGroupBox.Enabled = true;
                    RDataGroupBox.Enabled = true;
                    RequestGroupBox.Enabled = true;
                    break;

                case EikonDesktopDataAPI.EEikonStatus.Disconnected:
                    DisconnectFromEikon();
                    ConnectToEikonBtn.Enabled = true;
                    DisconnectFromEikonBtn.Enabled = false;
                    StatusPictureBox.Image = Properties.Resources.Red;
                    ToolStripStatusLabel1.Text = "Status : Disconnected from Eikon";
                    Dex2MgrGroupBox.Enabled = false;
                    RDataGroupBox.Enabled = false;
                    RequestGroupBox.Enabled = false;
                    break;

                case EikonDesktopDataAPI.EEikonStatus.LocalMode:
                    ReleaseDex2Mgr();
                    ConnectToEikonBtn.Enabled = true;
                    DisconnectFromEikonBtn.Enabled = false;
                    StatusPictureBox.Image = Properties.Resources.Orange;
                    ToolStripStatusLabel1.Text = "Status : Eikon local mode";
                    Dex2MgrGroupBox.Enabled = false;
                    RDataGroupBox.Enabled = false;
                    RequestGroupBox.Enabled = false;
                    break;

                case EikonDesktopDataAPI.EEikonStatus.Offline:
                    DisconnectFromEikon();
                    ConnectToEikonBtn.Enabled = true;
                    DisconnectFromEikonBtn.Enabled = false;
                    StatusPictureBox.Image = Properties.Resources.Red;
                    ToolStripStatusLabel1.Text = "Status : Offline";
                    Dex2MgrGroupBox.Enabled = false;
                    RDataGroupBox.Enabled = false;
                    RequestGroupBox.Enabled = false;
                    break;
            }

            UpdateDex2FormAccordingToDex2Status("","","");
        }




        public void ConnectToEikonBtn_Click(object sender, EventArgs e)
        {
            ToolStripStatusLabel1.Text = "Status : Connecting to Eikon...";
            ConnectToEikonBtn.Enabled = false;
            DisconnectFromEikonBtn.Enabled = true;

            ConnectToEikon();
        }

        private void DisconnectFromEikonBtn_Click(object sender, EventArgs e)
        {
            ToolStripStatusLabel1.Text = "Status : Disconnecting from Eikon...";
            ConnectToEikonBtn.Enabled = true;
            DisconnectFromEikonBtn.Enabled = false;

            DisconnectFromEikon();

            UpdateUserFormAccordingToConnectionStatus(EEikonStatus.Offline);
        }

        public void CreateDex2ManagerBtn_Click(object sender, EventArgs e)
        {
            CreateDex2Mgr();
            UpdateDex2FormAccordingToDex2Status("","","");
        }

        private void ReleaseDex2ManagerBtn_Click(object sender, EventArgs e)
        {
            ReleaseDex2Mgr();
            UpdateDex2FormAccordingToDex2Status("","","");
        }

        private void InstrumentsListTxt_TextChanged(object sender, EventArgs e)
        {
            MyRData = null;
            RequestGroupBox.Enabled = false;
            if (MyDex2Mgr != null && InstrumentsListTxt.TextLength != 0 && FieldsListTxt.TextLength != 0)
                CreateRDataBtn.Enabled = true;
            else
            {
                CreateRDataBtn.Enabled = false;
                //RequestGroupBox.Enabled = false;
            }
        }

        private void FieldsListTxt_TextChanged(object sender, EventArgs e)
        {
            MyRData = null;
            RequestGroupBox.Enabled = false;
            if (MyDex2Mgr != null && InstrumentsListTxt.TextLength != 0 && FieldsListTxt.TextLength != 0)
                CreateRDataBtn.Enabled = true;
            else
            {
                CreateRDataBtn.Enabled = false;
                //RequestGroupBox.Enabled = false;
            }
        }

        private void RequestParametersTxt_TextChanged(object sender, EventArgs e)
        {
            RequestGroupBox.Enabled = false;
            MyRData = null;
        }

        private void DisplayParametersTxt_TextChanged(object sender, EventArgs e)
        {
            RequestGroupBox.Enabled = false;
            MyRData = null;
        }

        private void CreateRDataBtn_Click(string argument,string dateStart,string dateEnd,EventArgs e)
        {
            string instrument="";
            string[] data=argument.Split('.');
            if(data[1]=="TO")
            {
                data[1]="XTSE";

            }
            else
            {
                data[1] = "XNYS";

            }
            if (data[0]== "Hi")
            {


                instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + data[1] + "'),BETWEEN(TR.Price52WeekHighDate," + dateEnd + "," + dateEnd + "))";
                //instrument = "SCREEN(U(IN(Equity(active,public,primary))),IN(TR.ExchangeMarketIdCode,'" + data[1] + "'),TR.PriceHigh>=TR.Price52WeekHigh)";
            }
            else
            {
                instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + data[1] + "'),BETWEEN(TR.Price52WeekLowDate," + dateEnd + "," + dateEnd + "))";

            }
            CreateRData(instrument, "TR.CommonName;TR.PriceClose;TR.PriceNetChg1D;TR.PricePctChg1D;TR.PriceHigh;TR.PriceLow;TR.Volume"
,"", "RH:In CH:Fd");

            //SCREEN(U(IN(Equity(active,public,primary))),IN(TR.ExchangeMarketIdCode,"XTSE"),TR.Price52WeekHighDate=20160422,CURN=USD)

            DataRetrievedTxt.Clear();

            if (MyRData != null)
                RequestGroupBox.Enabled = true;
            else
                RequestGroupBox.Enabled = false;
        }

        //----------------------------------------------------------------------
        private void UpdateUserFormAccordingToRDataResponse(Dex2.DEX2_DataStatus DataStatus, Object responseError)
        {
            DataRetrievedLbl.Text = "Data status: " + DataStatus.ToString();
            switch (DataStatus)
            {
                case DEX2_DataStatus.DE_DS_FULL:
                    long i = 0;
                    long j = 0;
                    Array l_response = MyRData.Data as Array;
                    Console.WriteLine("~Beginning of Data");

                    for( i =  l_response.GetLowerBound(0); i <=  l_response.GetUpperBound(0);++i)
                    {
                            long l_row;
                            l_row = i - l_response.GetLowerBound(0) + 1;
                            for( j = l_response.GetLowerBound(1); j <= l_response.GetUpperBound(1);++j)
                            {                              
                                // error handling could be done here
                                if (MyRData.Data[i, j] != null)
                                    Console.Write(MyRData.Data[i, j].ToString()+"\t");
                                //System.IO.StreamWriter file = new System.IO.StreamWriter("");
                                //System.IO.File.WriteAllText(@D:\\loghigh.txt, MyRData.Data[i, j].ToString() + "\n");

                            }
                            Console.WriteLine("");
                    }

                    Console.WriteLine("~End of Data");
                    this.Close();
                    break;


                case DEX2_DataStatus.DE_DS_NULL_EMPTY:
                    Console.WriteLine("~Beginning of Data");
                    if (responseError != null)
                        Console.WriteLine("Error : " + responseError);
                    else
                    {
                        Console.WriteLine("Error : DE_DS_NULL_EMPTY");
                    }
                    Console.WriteLine("~End of Data");
                    this.Close();
                    break;

                case DEX2_DataStatus.DE_DS_NULL_ERROR:

                    Console.WriteLine("~Beginning of Data");
                    if (responseError != null)
                        Console.WriteLine("Error : " + responseError);
                    else
                    {
                        Console.WriteLine("Error : DE_DS_NULL_ERROR");
                    }
                    Console.WriteLine("~End of Data");
                    this.Close();
                    break;

                case DEX2_DataStatus.DE_DS_NULL_TIMEOUT:
                    Console.WriteLine("~Beginning of Data");
                    if (responseError != null)
                        Console.WriteLine("Error : " + responseError);
                    else
                    {
                        Console.WriteLine("Error : DE_DS_NULL_TIMEOUT");
                    }
                    Console.WriteLine("~End of Data");
                    this.Close();
                    break;

                case DEX2_DataStatus.DE_DS_PARTIAL:
                    Console.WriteLine("~Beginning of Data");
                    if (responseError != null)
                        Console.WriteLine("Error : " + responseError);
                    else
                    {
                        Console.WriteLine("Error : DE_DS_PARTIAL");
                    }
                    Console.WriteLine("~End of Data");
                    this.Close();
                    break;

            }


            CreateRDataBtn.Enabled = true;
            RequestDataBtn.Enabled = true;
        }

        private void RequestDataBtn_Click(object sender, EventArgs e)
        {
            RequestDataBtn.Enabled = false;
            CreateRDataBtn.Enabled = false;
            DataRetrievedLbl.Text = "Retrieve Data...";

            RequestDex2Data();
        }

        private void ClearRetrievedDataBtn_Click(object sender, EventArgs e)
        {
            DataRetrievedTxt.Clear();
        }

        private void EgInstrumentsListBtn_Click(object sender, EventArgs e)
        {
            InstrumentsListTxt.Text = "MDT.N;KSS.N;NI.N";
        }

        private void EgFieldsListBtn_Click(object sender, EventArgs e)
        {
            FieldsListTxt.Text = "TR.Revenue;TR.EBITDA";
        }

        private void EgRequesParamsBtn_Click(object sender, EventArgs e)
        {
            RequestParametersTxt.Text = "Period=FY-5";
        }

        private void EgDisplayParamsBtn_Click(object sender, EventArgs e)
        {
            DisplayParametersTxt.Text = "RH:In CH:Fd";
        }

        private void RDataGroupBox_Enter(object sender, EventArgs e)
        {

        }
    }
}

dgarrard

Hello @jason.ramchandani01

Was the explanation and code I provided sufficient to allow you to test this?
David

jason.ramchandani01

@dgarrard Apols for delay in getting back to you - I checked out the screener query and it works for say yesterday or any date before so its not a problem with the query per se - just when it is being run. I am not sure why this might have changed (if it indeed has) for the two exchanges you mention. Can I also ask what timezone you are in? When you say 5PM and 6AM do you mean 5PM on the day and then 6AM in the morning of the next day? I have opened a case with the content team responsible on your behalf and will wait for them to get back to me. The case number is 10738822. Please bear with us and I will report back here.

dgarrard

Thank you! This is running on Eastern time. Location is Montreal Canada.

Could you tell me what time you ran it successfully and I will test from their PC at that same time tomorrow.
David

jason.ramchandani01

@dgarrard I ran this at 10:49am London which is 05:49 Montreal (so slightly before your 6am). I ran the screener query in Python but it uses the same backend service as the older API you are using.

instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + "XNYS" + "'),BETWEEN(TR.Price52WeekHighDate," + "20211201" + "," + "20211201" + "))"

ek.get_data(instrument,'TR.PRICECLOSE')

I then ran it for TSE at 10:54 London (05:54 Montreal):

instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + "XTSE" + "'),BETWEEN(TR.Price52WeekHighDate," + "20211201" + "," + "20211201" + "))"

ek.get_data(instrument,'TR.PRICECLOSE')

Are you able to replicate this?

dgarrard

Thank you Jason,

Could you send me the python script, please? I would like to run it on the client's machine. I am now wondering if this could be some licencing issue.

David

jason.ramchandani01

@dgarrard You should 'pip install refinitiv.dataplatform' library then generate an App Key using the APPKEY (type appkey into Eikon search bar) :

import refinitiv.dataplatform.eikon as ek
ek.set_app_key('YOUR APPKEY HERE')

instrument = "SCREEN(U(IN(Equity(active,public,primary,countryprimaryquote))),IN(TR.ExchangeMarketIdCode,'" + "XTSE" + "'),BETWEEN(TR.Price52WeekHighDate," + "20211201" + "," + "20211201" + "))"

df, err = ek.get_data(instrument,['TR.PRICECLOSE.date','TR.PRICECLOSE'])
df

A good return will return a dataframe (df), otherwise you can interrogate the err object. Let me know how you get on. FYI if you are using .NET you could try our Refinitiv Data Platform (RDP) APIs & libraries in .NET - they would be much simpler to use and provide many more capabilities for most use cases. Let me know how you get on.

dgarrard

I ran the script that I shared above just now to confirm the failure and got this in the command prompt:

Does the SD_Bootstrap error block help you diagnose this for me?

David

jason.ramchandani01

@dgarrard were you able to run the python script? This is the simplest way to see if its a permissioning thing or not.

jason.ramchandani01

@dgarrard I ran this just now and can get a return :

Are you able to run any simple other requests on the system using that API? such as getting any fundamental or pricing data? Lets try this -can you open the (Data Item Browser app - type DIB into Eikon Search bar) then look for the TR.Price52WeekHighDate field and also add any instrument from the above return eg 'BNS.TO' - do you see the correct value being returned there (note I had to actually click on the field to get the value to populate there)?

dgarrard

Yes I am seeing the correct data in the DIB for BNS.TO

dgarrard

Here is a screenshot

dgarrard

Unfortunately, it appears that python is not installed on this clients PC.

jason.ramchandani01

@dgarrard not really - its not a component of ours so I won't be able to help there. See what your programmer says on weekend. The way to zero in on this is to try to run a simple query using dex2 and see if that works - my thinking is that nothing might be working which would lead to some connection issue - which is usually a change in firewall settings or similar. The code has not changed - with the DIB you have seen the data is available, my python scripts show the screener query is working as expected - so most likely connection/firewall issue imo.

dgarrard

Hi jason.ramchandani01 It appears that there was some change in the data format being returned by the old API call.

Per your recommendation, I installed and migrated this call to Python. The filter returns the expected dataframe and I am now in final testing.

I think I saw that you said another case was opened. That can be closed also.

Thank you very much,

David

jason.ramchandani01

@dgarrard OK great you were able to do that - just in terms of completeness diid your programmer tell you what in the data format has been changed in the return - as I am unaware of this (its an older api) but would like to let the dev team know. Thanks I will close the content case as well.

EXPLORE OUR SITES