Eikon API seems Instable - Bad Request Error and Inconsistent Data Retrieval

Hi there,

I am currently testing the Eikon API for the first time so I do not have much experience with it yet. I put together a little python function for testing, which is meant to give me data for a list of stocks and hence returns identifiers, names, and some fundamental data. The code I used for this is attached below, which currently uses the constituent of the SXXP Index to pull some data (individual app key needs to be provided)...

Now when running this file I bump into 2 problems sometimes:

1.) Eikon Request gives me error, even for small
requests

Sometimes
the code runs into an error out of nowhere (sometimes it works just
fine). I get the following error message, which is basically an error code 400,
“Bad Request Error” coming from the “json_requestl.py” file and the function
“check_server_error”.
Why do those server errors occur? I initially thought (and by reading through some other posts here) I thought the request was too large in one go, but even changing to the DAX universe with 30 names bumps into this problem occasionaly...

2.) Eikon gives me
weird behavior when changing input fields

The following function (which is also comes from the code below) let's me pull RIC's and Names for the constituents of the SXXP as of Jan 2017:

inst_list, _ =
ek.get_data(instruments=['.STOXX'],
fields=['TR.IndexConstituentRIC', 'TR.IndexConstituentName'],
parameters={'SDate': '2017-01-15', 'EDate': '2017-01-15'})

when continuing a statement after this, I can pull other data using the obtained RIC's, where I can use TR.ISINCode or TR.Sedol to get two different further identifiers (just as an example here). When running the fields individually, I get a complete list of identifiers for each constituent.

isin_list, _ =
ek.get_data(instruments=inst_list.loc[:,'Constituent
RIC'], fields=['TR.ISINCode'])

this is all cool so far, however, the problem comes when I try to pull ISIN and Sedol at the same time. Again, this doesnt happen all the time and just occaionally, I get an incomplete list of the two identifiers like the one below, where ISINs and/or SEDOLs are missing sometimes, where I would get some when I run this individually for each field.

So far, me pulling data feels quite instable. Once I bump into these problems, the only thing that seems to resolve this is me closing and re-opening the Python IDE. (I'm on Pycharm btw.)

Would you be able to help on this/would you know what I am doing wrong?

Many thanks.

Best,

Chris

import eikon as ek

def get_eikon_data_for_index(universe, dt, eikon_key=None, ric_codes=None, fund_flds=None, fperiod=None, isin=False, xl_name=None):

    if sum(x is None for x in [fund_flds, fperiod]) == 1:
        ValueError('Fundamental fields and list must both be defined!')

    if eikon_key is None:
        eikon_key = 'Need to set own key here...'
    ek.set_app_key(eikon_key)

    if ric_codes is None:
        static_flds = ['TR.IndexConstituentRIC', 'TR.IndexConstituentName']
        inst_list, err_const = ek.get_data(instruments=universe, fields=static_flds,
                                           parameters={'SDate': dt, 'EDate': dt})
    else:
        inst_list = ric_codes

    # get the list of stocks within the STOXX 600 universe...
    if isin:
        isin_list, err_isin = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(), fields=['TR.ISINCode', 'TR.SEDOL'])
        inst_list[isin_list.columns[1:]] = isin_list.iloc[:, 1:]

    if fperiod is not None:
        fund_data, err_fund = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(), fields=fund_flds, parameters={'SDate': dt, 'Period': fperiod})
    else:
        fund_data, err_fund = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(), fields=fund_flds, parameters={'SDate': dt})
    inst_list[fund_data.columns[1:]] = fund_data.iloc[:, 1:]

    # write this over into a spreadsheet
    if xl_name is not None:
        inst_list.to_excel(xl_name)

    return inst_list


if __name__== '__main__':

    date = '2017-01-15'
    univ = ['.STOXX']

    # get some fundamentals here...
    fund_fields = ['TR.EPSMean', 'TR.OperatingMarginPercent']
    f_per = 'FY0'
    fund_data = get_eikon_data_for_index(univ, date, fund_flds=fund_fields, fperiod=f_per, isin=True)

    # get dividend yield data...
    div_yld = ['TR.DPSMeanYield']
    f_per = 'FY1'
    div_yld = get_eikon_data_for_index(univ, date, ric_codes=fund_data.loc[:, ['Constituent RIC']], fund_flds=div_yld, fperiod=f_per, isin=False)

    # get Net Debt/EBITDA data...
    n_d_to_ebitda = ['TR.NetDebtToEBITDA']
    net_debt_to_ebitda = get_eikon_data_for_index(univ, date, ric_codes=fund_data.loc[:, ['Constituent RIC']],fund_flds=n_d_to_ebitda, isin=False)

    #combine data...
    fund_data[div_yld.columns[1:]] = div_yld.iloc[:, 1:]
    fund_data[net_debt_to_ebitda.columns[1:]] = net_debt_to_ebitda.iloc[:, 1:]

    # write to excel...
    fund_data.to_excel('test_point-in-time.xlsx')

Find more posts tagged with

python

eikon

workspace

eikon-data-api

refinitiv-dataplatform-eikon

workspace-data-api

Accepted answers

Alex Putkov.1

@christoph.auth
I've been able to reproduce the issue with most ISINs and some SEDOLs missing in the response using the test script you provided. It's a rather strange issue as it only happens once or twice for every 100 requests, which are all identical.
What I suggest you do is replace TR.ISINCode with TR.ISIN in your request. TR.ISINCode is a legacy field mnemonic, which is kept for backwards compatibility. It should not be used in any new applications or scripts. Once I replaced TR.ISINCode with TR.ISIN I was no longer able to reproduce the issue with missing ISINs. I also stopped experiencing "Backend error. 400 Bad Request" for this particular request (get ISINs and SEDOLS for 601 historical constituents of .STOXX). I see that the response from the Web service is noticeably faster when using TR.ISIN instead of TR.ISINCode, which explains why I stopped experiencing "Backend error. 400 Bad Request". This error is still being returned for other requests including the request for historical constituents of .STOXX.

All comments

chavalit-jintamalit

Hi @christoph.auth

Can you attach your py code in a text attachment ?

Thanks.

christoph.auth

Sure, see below. Let me know if this didn't work.

Many thanks.

eikon-api-example-code.txt

chavalit-jintamalit

I could reproduce the "Backend error. 400 Bad Request" 1 time from my several test runs.

Let me try to get some information and update you again.

This is my test code:

import time
import eikon as ek
ek.set_app_key('<your key>')
static_flds = ['TR.IndexConstituentRIC', 'TR.IndexConstituentName']
for x in range(100):
   print('round: ',x)
   inst_list, err_const = ek.get_data(instruments=['.STOXX'], fields=static_flds, parameters={'SDate': '2017-01-15', 'EDate': '2017-01-15'})
   print(err_const)
   print(inst_list.tail(1))
   print()
   time.sleep(5)
print('end')

So I just loop your first API call and wait for 5 seconds each loop.

Alex Putkov.1

@christoph.auth
"Backend error. 400 Bad Request" is a generic error. Most commonly it happens due to the timeout at the backend. It may happen even for seemingly small requests, although the bigger the request the more likely this error. Resubmitting the exact same request may result in the data being returned. There's work being undertaken to address this issue and we hope it will be resolved shortly.
As for the problem where not all identifiers are returned in the response, I've not heard anyone experiencing this before. This issue needs to be investigated and to be able to investigate it, we need to reproduce it first. Any info you can provide that would help us reproduce the issue, any patterns you noticed etc would be much appreciated.

Next 1.1.0 version of Eikon Data API will fix some cases of timeout in back-end intermediate services used by DataGrid endpoint.
It should fix most of "Backend error. 400 Bad Request" raised when user call get_data function.

QA is currently validating this version.
It's a question of weeks to publish it.

christoph.auth

Thanks @Alex Putkov.1 and @pierre.faurel for this and for putting a time line on this as well.

In general, the "Bad Request" error just seems to happen quite often for me, so re-submitting data until it works can in theory become quite time consuming unfotunately. This is not a problem for now, but our systems are set up to run overnight and have a certain deadline by which this needs to be done (and if data gathering takes too long or is stuck, processes before and after that will also fail so this might not be so good unfortunately).

So to "fix"/re-submit the data request and try to replicate the second strange behaviour with ISIN's, I used the logic from @chavalit-jintamalit (thanks for this btw.) from above:


import time
import eikon as ek
ek.set_app_key('In here with your app key...')
static_flds = ['TR.IndexConstituentRIC', 'TR.IndexConstituentName']
for x in range(100):
    print('round: ', x)
    err_const = 0 # random stuff
    err_isin = 0 # random stuff again
    while err_const is not None:
        try:
            inst_list, err_const = ek.get_data(instruments=['.STOXX'], fields=static_flds,parameters={'SDate': '2017-01-15','EDate': '2017-01-15'})
        except:
            pass

    while err_isin is not None:
        try:
            isin_list, err_isin = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(),fields=['TR.ISINCode', 'TR.SEDOL'])
        except:
            pass
    if (isin_list['ISIN Code'].values == '').sum() > 0:
        print(isin_list.head(5))
        print('Weird stuff going on in this loop!')
    print()
    time.sleep(2)
print('end')

When doing this, it takes a bit of time but eventually (for me in loop 68) there are a lot of ISINs missing (usually there is an ISIN for all stocks available in the universe). You also see that the Bad Request error happens quite often for me as well...

Many thanks for your help.

Best,

Chris

christoph.auth

I don't seem to be very good with posting code in here somehow...

There is a .txt file attached as well with this...

eikon-api-example-code.txt

Alex Putkov.1

christoph.auth

Thanks a lot @Alex Putkov.1

So I suppose there isn't any way for me to figure out if a field is a legacy field or not and, if I find two fields supposedly giving me the same info, which of them I should use then? (my standard way of finding data fields is via the Data Item Browser so far, not sure if there is a better way to do this?!)

EXPLORE OUR SITES