Discover Refinitiv
MyRefinitiv Refinitiv Perspectives Careers
Created with Sketch.
All APIs Questions & Answers  Register |  Login
Ask a question
  • Questions
  • Tags
  • Badges
  • Unanswered
Search:
  • Home /
  • Eikon Data APIs /
avatar image
Question by christoph.auth · Apr 29, 2019 at 08:27 AM · eikon data apibad requestinstable

Eikon API seems Instable - Bad Request Error and Inconsistent Data Retrieval

Hi there,

I am currently testing the Eikon API for the first time so I do not have much experience with it yet. I put together a little python function for testing, which is meant to give me data for a list of stocks and hence returns identifiers, names, and some fundamental data. The code I used for this is attached below, which currently uses the constituent of the SXXP Index to pull some data (individual app key needs to be provided)...

Now when running this file I bump into 2 problems sometimes:

1.) Eikon Request gives me error, even for small requests

  • Sometimes the code runs into an error out of nowhere (sometimes it works just fine). I get the following error message, which is basically an error code 400, “Bad Request Error” coming from the “json_requestl.py” file and the function “check_server_error”.
  • Why do those server errors occur? I initially thought (and by reading through some other posts here) I thought the request was too large in one go, but even changing to the DAX universe with 30 names bumps into this problem occasionaly...

2.) Eikon gives me weird behavior when changing input fields

  • The following function (which is also comes from the code below) let's me pull RIC's and Names for the constituents of the SXXP as of Jan 2017:

inst_list, _ = ek.get_data(instruments=['.STOXX'], fields=['TR.IndexConstituentRIC', 'TR.IndexConstituentName'], parameters={'SDate': '2017-01-15', 'EDate': '2017-01-15'})

  • when continuing a statement after this, I can pull other data using the obtained RIC's, where I can use TR.ISINCode or TR.Sedol to get two different further identifiers (just as an example here). When running the fields individually, I get a complete list of identifiers for each constituent.

isin_list, _ = ek.get_data(instruments=inst_list.loc[:,'Constituent RIC'], fields=['TR.ISINCode'])

  • this is all cool so far, however, the problem comes when I try to pull ISIN and Sedol at the same time. Again, this doesnt happen all the time and just occaionally, I get an incomplete list of the two identifiers like the one below, where ISINs and/or SEDOLs are missing sometimes, where I would get some when I run this individually for each field.

So far, me pulling data feels quite instable. Once I bump into these problems, the only thing that seems to resolve this is me closing and re-opening the Python IDE. (I'm on Pycharm btw.)

Would you be able to help on this/would you know what I am doing wrong?

Many thanks.

Best,

Chris

import eikon as ek

def get_eikon_data_for_index(universe, dt, eikon_key=None, ric_codes=None, fund_flds=None, fperiod=None, isin=False, xl_name=None):

    if sum(x is None for x in [fund_flds, fperiod]) == 1:
        ValueError('Fundamental fields and list must both be defined!')

    if eikon_key is None:
        eikon_key = 'Need to set own key here...'
    ek.set_app_key(eikon_key)

    if ric_codes is None:
        static_flds = ['TR.IndexConstituentRIC', 'TR.IndexConstituentName']
        inst_list, err_const = ek.get_data(instruments=universe, fields=static_flds,
                                           parameters={'SDate': dt, 'EDate': dt})
    else:
        inst_list = ric_codes

    # get the list of stocks within the STOXX 600 universe...
    if isin:
        isin_list, err_isin = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(), fields=['TR.ISINCode', 'TR.SEDOL'])
        inst_list[isin_list.columns[1:]] = isin_list.iloc[:, 1:]

    if fperiod is not None:
        fund_data, err_fund = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(), fields=fund_flds, parameters={'SDate': dt, 'Period': fperiod})
    else:
        fund_data, err_fund = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(), fields=fund_flds, parameters={'SDate': dt})
    inst_list[fund_data.columns[1:]] = fund_data.iloc[:, 1:]

    # write this over into a spreadsheet
    if xl_name is not None:
        inst_list.to_excel(xl_name)

    return inst_list


if __name__== '__main__':

    date = '2017-01-15'
    univ = ['.STOXX']

    # get some fundamentals here...
    fund_fields = ['TR.EPSMean', 'TR.OperatingMarginPercent']
    f_per = 'FY0'
    fund_data = get_eikon_data_for_index(univ, date, fund_flds=fund_fields, fperiod=f_per, isin=True)

    # get dividend yield data...
    div_yld = ['TR.DPSMeanYield']
    f_per = 'FY1'
    div_yld = get_eikon_data_for_index(univ, date, ric_codes=fund_data.loc[:, ['Constituent RIC']], fund_flds=div_yld, fperiod=f_per, isin=False)

    # get Net Debt/EBITDA data...
    n_d_to_ebitda = ['TR.NetDebtToEBITDA']
    net_debt_to_ebitda = get_eikon_data_for_index(univ, date, ric_codes=fund_data.loc[:, ['Constituent RIC']],fund_flds=n_d_to_ebitda, isin=False)

    #combine data...
    fund_data[div_yld.columns[1:]] = div_yld.iloc[:, 1:]
    fund_data[net_debt_to_ebitda.columns[1:]] = net_debt_to_ebitda.iloc[:, 1:]

    # write to excel...
    fund_data.to_excel('test_point-in-time.xlsx')
bad-request-error.png (42.1 KiB)
sedol-isin-example.png (33.7 KiB)

People who like this

0 Show 4
Comment
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
REFINITIV
chavalit.jintamalit ♦♦ · Apr 29, 2019 at 08:51 AM 0
Share

Hi @christoph.auth

Can you attach your py code in a text attachment ?

Thanks.

avatar image
christoph.auth chavalit.jintamalit ♦♦ · Apr 29, 2019 at 08:55 AM 0
Share

Sure, see below. Let me know if this didn't work.

Many thanks.

eikon-api-example-code.txt

eikon-api-example-code.txt (3.0 KiB)
avatar image
REFINITIV
chavalit.jintamalit ♦♦ christoph.auth · Apr 29, 2019 at 10:39 AM 0
Share

Hi @christoph.auth

From my understanding, you have 3 consecutive calls in get_eikon_data_for_index method.

The 1st call get list of RIC.

The 2nd call get more info from the result of 1st call.

And the 3rd call get fund_fields from the result of 1st call.

I have tried running the code for 5 times but I cannot see the error.

Is it possible that you log the request which it make this error happen ?

4 Replies

  • Sort: 
avatar image
REFINITIV
Best Answer
Answer by Alex Putkov. · May 01, 2019 at 06:25 PM

@christoph.auth
I've been able to reproduce the issue with most ISINs and some SEDOLs missing in the response using the test script you provided. It's a rather strange issue as it only happens once or twice for every 100 requests, which are all identical.
What I suggest you do is replace TR.ISINCode with TR.ISIN in your request. TR.ISINCode is a legacy field mnemonic, which is kept for backwards compatibility. It should not be used in any new applications or scripts. Once I replaced TR.ISINCode with TR.ISIN I was no longer able to reproduce the issue with missing ISINs. I also stopped experiencing "Backend error. 400 Bad Request" for this particular request (get ISINs and SEDOLS for 601 historical constituents of .STOXX). I see that the response from the Web service is noticeably faster when using TR.ISIN instead of TR.ISINCode, which explains why I stopped experiencing "Backend error. 400 Bad Request". This error is still being returned for other requests including the request for historical constituents of .STOXX.

Comment
christoph.auth

People who like this

1 Show 2 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
christoph.auth · May 06, 2019 at 03:36 AM 0
Share

Thanks a lot @Alex Putkov.

So I suppose there isn't any way for me to figure out if a field is a legacy field or not and, if I find two fields supposedly giving me the same info, which of them I should use then? (my standard way of finding data fields is via the Data Item Browser so far, not sure if there is a better way to do this?!)

avatar image
REFINITIV
Alex Putkov. ♦♦ christoph.auth · May 06, 2019 at 04:02 PM 0
Share

You're right. I'm afraid there's no way for you to know if a field is legacy or not. Data Item Browser or Formula Builder in Excel are the best ways to discover metadata for use in Eikon Data APIs. There's no better way. If you find two fields returning similar results and you're in doubt which field you should use, the best resource is Refinitiv Helpdesk, which you can reach by either calling the Helpdesk number in your country or by using Contact Us capability in Eikon application.

avatar image
REFINITIV
Answer by chavalit.jintamalit · Apr 29, 2019 at 12:32 PM

I could reproduce the "Backend error. 400 Bad Request" 1 time from my several test runs.

Let me try to get some information and update you again.

This is my test code:

import time
import eikon as ek
ek.set_app_key('<your key>')
static_flds = ['TR.IndexConstituentRIC', 'TR.IndexConstituentName']
for x in range(100):
   print('round: ',x)
   inst_list, err_const = ek.get_data(instruments=['.STOXX'], fields=static_flds, parameters={'SDate': '2017-01-15', 'EDate': '2017-01-15'})
   print(err_const)
   print(inst_list.tail(1))
   print()
   time.sleep(5)
print('end')

So I just loop your first API call and wait for 5 seconds each loop.

Comment
christoph.auth

People who like this

1 Show 0 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
REFINITIV
Answer by Alex Putkov. · Apr 29, 2019 at 04:56 PM

@christoph.auth
"Backend error. 400 Bad Request" is a generic error. Most commonly it happens due to the timeout at the backend. It may happen even for seemingly small requests, although the bigger the request the more likely this error. Resubmitting the exact same request may result in the data being returned. There's work being undertaken to address this issue and we hope it will be resolved shortly.
As for the problem where not all identifiers are returned in the response, I've not heard anyone experiencing this before. This issue needs to be investigated and to be able to investigate it, we need to reproduce it first. Any info you can provide that would help us reproduce the issue, any patterns you noticed etc would be much appreciated.

Comment
christoph.auth

People who like this

1 Show 1 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
REFINITIV
pierre.faurel · Apr 30, 2019 at 04:57 AM 1
Share

Next 1.1.0 version of Eikon Data API will fix some cases of timeout in back-end intermediate services used by DataGrid endpoint.
It should fix most of "Backend error. 400 Bad Request" raised when user call get_data function.

QA is currently validating this version.
It's a question of weeks to publish it.

avatar image
Answer by christoph.auth · Apr 30, 2019 at 05:29 AM

Thanks @Alex Putkov. and @pierre.faurel for this and for putting a time line on this as well.

In general, the "Bad Request" error just seems to happen quite often for me, so re-submitting data until it works can in theory become quite time consuming unfotunately. This is not a problem for now, but our systems are set up to run overnight and have a certain deadline by which this needs to be done (and if data gathering takes too long or is stuck, processes before and after that will also fail so this might not be so good unfortunately).

So to "fix"/re-submit the data request and try to replicate the second strange behaviour with ISIN's, I used the logic from @chavalit.jintamalit (thanks for this btw.) from above:

import time
import eikon as ek
ek.set_app_key('In here with your app key...')
static_flds = ['TR.IndexConstituentRIC', 'TR.IndexConstituentName']
for x in range(100):
    print('round: ', x)
    err_const = 0 # random stuff
    err_isin = 0 # random stuff again
    while err_const is not None:
        try:
            inst_list, err_const = ek.get_data(instruments=['.STOXX'], fields=static_flds,parameters={'SDate': '2017-01-15','EDate': '2017-01-15'})
        except:
            pass

    while err_isin is not None:
        try:
            isin_list, err_isin = ek.get_data(instruments=inst_list.loc[:, 'Constituent RIC'].tolist(),fields=['TR.ISINCode', 'TR.SEDOL'])
        except:
            pass
    if (isin_list['ISIN Code'].values == '').sum() > 0:
        print(isin_list.head(5))
        print('Weird stuff going on in this loop!')
    print()
    time.sleep(2)
print('end')

When doing this, it takes a bit of time but eventually (for me in loop 68) there are a lot of ISINs missing (usually there is an ISIN for all stocks available in the universe). You also see that the Bad Request error happens quite often for me as well...

Many thanks for your help.

Best,

Chris


replicating-error-messages-eikon.jpg (23.0 KiB)
Comment

People who like this

0 Show 1 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
christoph.auth · Apr 30, 2019 at 05:32 AM 0
Share

I don't seem to be very good with posting code in here somehow... ;)

There is a .txt file attached as well with this...

eikon-api-example-code.txt

eikon-api-example-code.txt (1.0 KiB)

Watch this question

Add to watch list
Add to your watch list to receive emailed updates for this question. Too many emails? Change your settings >
10 People are following this question.

Related Questions

400 Bad Request though working before

How to expand a RIC chain?

Eikon Python API Error code 503

how to get the PE of s&p 500

Eikon Python API Empty Response for Private Company Financial Data - e.g. TR.Revenue, TR.GrossProfit

  • Feedback
  • Copyright
  • Cookie Policy
  • Privacy Statement
  • Terms of Use
  • Careers
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Alpha
  • App Studio
  • Block Chain
  • Bot Platform
  • Calais
  • Connected Risk APIs
  • DSS
  • Data Fusion
  • Data Model Discovery
  • Datastream
  • Eikon COM
  • Eikon Data APIs
  • Elektron
    • EMA
    • ETA
    • WebSocket API
  • Legal One
  • Messenger Bot
  • Messenger Side by Side
  • ONESOURCE
    • Indirect Tax
  • Open PermID
    • Entity Search
  • Org ID
  • PAM
    • PAM - Logging
  • ProView
  • ProView Internal
  • Product Insight
  • Project Tracking
  • Refinitiv Data Platform
    • Refinitiv Data Platform Libraries
  • Rose's Space
  • Screening
    • Qual-ID API
    • Screening Deployed
    • Screening Online
    • World-Check One
    • World-Check One Zero Footprint
  • Side by Side Integration API
  • TR Knowledge Graph
  • TREP APIs
    • CAT
    • DACS Station
    • Open DACS
    • RFA
    • UPA
  • TREP Infrastructure
  • TRIT
  • TRKD
  • TRTH
  • Thomson One Smart
  • Transactions
    • REDI API
  • Velocity Analytics
  • Wealth Management Web Services
  • World-Check Data File
  • Explore
  • Tags
  • Questions
  • Badges