How do I fix this error with data field 'TR.RNSARPctOSPostTrans'?

My code, which is intended to append new data daily to an already existing csv file named 'Daily API Download Master.csv:

#import packages

import eikon as ek # the Eikon Python wrapper package

import pandas as pd

import numpy as np

import datetime

from datetime import timedelta, date, datetime

from pandas.tseries.offsets import BDay


#connects to Bill's Eikon terminal

ek.set_app_key('xxxx')


#Retreive the data fields from the Eikon API using dynamic dates for a list of RICs

df,e = ek.get_data("lists('Inv Trust List')","TR.RIC")

ric_list = df['Instrument'].tolist()

df, e = ek.get_data(ric_list,

['TR.RNSFilerName',

'TR.RNSAnnouncedDate',

'TR.RNSTransactionType',

'TR.RNSARNumShrsTransacted',

'TR.RNSARPctOSTransacted',

'TR.RNSARTransactionPrice',

'TR.RNSARMktValTransaction',

'TR.RNSARTotShrsPostTrans',

'TR.RNSARPctOSPostTrans'])


end_date = date.today()

start_date = end_date - BDay(1)


end_date_str = datetime.strftime(end_date, "%Y-%m-%d")

start_date_str = datetime.strftime(start_date, "%Y-%m-%d")

df['RNS Announced Date'] = pd.to_datetime(df['RNS Announced Date'])

mask = (df['RNS Announced Date'] > start_date_str) & (df['RNS Announced Date'] <= end_date_str)

df = df.loc[mask]


df.rename(columns={'RNS AR Price (at Transaction) - £': 'RNS AR Price (at Transaction) GBP',

'RNS AR Market Value of Transaction - £': 'RNS AR Market Value of Transaction - GBP'},

inplace=True)


#append new csv to Master

df.to_csv('Daily API Download Master.csv', mode='a', header=False)

Backend error. 400 Bad Request


---------------------------------------------------------------------------
EikonError Traceback (most recent call last)
<ipython-input-16-f4c48e9e9e88> in <module>()
22 'TR.RNSARMktValTransaction',
23 'TR.RNSARTotShrsPostTrans',
---> 24 'TR.RNSARPctOSPostTrans'])
25 26 end_date = date.today()

~\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\data_grid.py in get_data(instruments, fields, parameters, field_name, raw_output, debug)
184 payload = {'requests': [payload]}
185 --> 186 result = eikon.json_requests.send_json_request(_endpoint, payload, debug=debug)
187 188 if result.get('responses'):

~\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py in send_json_request(entity, payload, debug)
116 ticket = _check_ticket_async(result)
117 --> 118 _check_server_error(result)
119 return result
120 else:

~\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py in _check_server_error(server_response)
192 status, reason_phrase, version, content, headers = error_message.split(',')[:5]
193 logger.error(error_message)
--> 194 raise EikonError(int(server_response['ErrorCode']), error_message)
195 196 # check DataGrid response (server response is JSON and it can contain error + transactionId keys)

EikonError: Error code 400 | Backend error. 400 Bad Request


Best Answer

  • bill39
    bill39 Explorer
    Answer ✓

    Thanks I have checked and I have version 1.0.2 so based on your above comment that isn't the issue. I have reduced my list to 8 securities and it has worked so it seems as you suspected it is likely the number of instruments that is the problem. Please would you provide the example that you have kindly offered of how to slice, loop and concatenate the list?

Answers

  • Alex Putkov.1
    Alex Putkov.1 ✭✭✭✭✭

    This is a generic backend error. Are you able to consistently reproduce it or does it happen randomly? One scenario when this error may occur is if the request is too large and it gets timed out from the backend. How long is the list of instruments in your request? Are you able to reproduce the issue with a single instrument or a small handful? Finally make sure that you use Eikon Data APIs library for Python v1.0.1 or higher. To verify which version you're using execute ek.__version__ command.

  • zoya faberov
    zoya faberov ✭✭✭✭✭

    Hello @bill39,

    This same code worked for me, with a couple of modifications.

    1. Do not have your instrument list, so have created mine for testing:

    df,e = ek.get_data("lists('ZF_RNS')","TR.RIC")
    df

    InstrumentRIC0AAIF.LAAIF.L1KEY.TOKEY.TO2BIV.PBIV.P3SILV


    ric_list = df['Instrument'].tolist()
    df, e = ek.get_data(ric_list,
    ['TR.RNSFilerName',
    'TR.RNSAnnouncedDate',
    'TR.RNSTransactionType',
    'TR.RNSARNumShrsTransacted',
    'TR.RNSARPctOSTransacted',
    'TR.RNSARTransactionPrice',
    'TR.RNSARMktValTransaction',
    'TR.RNSARTotShrsPostTrans',
    'TR.RNSARPctOSPostTrans'])
    df

    2. I did not get any RNS outputs within 1 day, so increased the interval

    end_date = date.today()
    start_date = end_date - BDay(1000)
    end_date_str = datetime.strftime(end_date, "%Y-%m-%d")
    start_date_str = datetime.strftime(start_date, "%Y-%m-%d")
    df['RNS Announced Date'] = pd.to_datetime(df['RNS Announced Date'])
    mask = (df['RNS Announced Date'] > start_date_str) & (df['RNS Announced Date'] <= end_date_str)
    df = df.loc[mask]
    df


    df.rename(columns={'RNS AR Price (at Transaction) - £': 'RNS AR Price (at Transaction) GBP',
    'RNS AR Market Value of Transaction - £': 'RNS AR Market Value of Transaction - GBP'},
    inplace=True)
    df
    df.to_csv('Daily API Download Master.csv', mode='a', header=Fals

    For me, what I see on my intermediate output of df, is what gets saved into the csv file.

    If you like, try with this tiny instrument list, see if it works as expected.

    If it does, the next step will be to start adding your instruments back to see which instrument perhaps has a value returned, that is resulting in the error.

    If it does not, I would look for environmental difference, for example, try to verify that all the libraries are up to date, folder is permissioned for writing.

    Do not see an issue with code, so would look in the direction of specific content and environment.

  • bill39
    bill39 Explorer

    It worked on the first day I created the programme but since then I keep getting the same backend error. My list of RICs contains 574 securities so perhaps it is too large. How would I write the programme to stop it timing out / to deal with the large number of securities I need the data for?

    Additionally, when entering ek.__version__ command directly into my command prompt I get the following message:

    'ek.__version__' is not recognized as an internal or external command, operable program or batch file.

    Where do I type the command in order to verify my version?

  • bill39
    bill39 Explorer

    My list of RICs has 574 securities so it sounds like that is the problem. Do you know how I can rewrite the code so that the request doesn't time out, please?

  • zoya faberov
    zoya faberov ✭✭✭✭✭

    Hi @bill39,

    I agree with @Alex Putkov.1

    A quick test is to create a really small list

    image

    and request with that new list, instead of "Inv Trust List"

    This should give you additional information on where the issue is or is not.

    Anywhere from python is good to run ek.__version__

    So either start python interactive prompt, add to python script or to python notebook.

    image

  • Alex Putkov.1
    Alex Putkov.1 ✭✭✭✭✭

    For this type of request the list of 574 instruments is likely too large. I suggest you slice the list, retrieve the data separately for each slice in a loop and then concatenate the dataframes. Let me know if you need an example. The optimal size of the slice can only be established empirically. I suggest starting with 50 instruments, and then increasing or decreasing it depending on the results. The larger the size of the slice, the less requests you need to send, the less the overall runtime of your script (although the relationship is not linear at all), but the more the chance of a timeout.
    For checking the version of Eikon Data APIs for Python library, which I strongly suggest you do to ensure it's at least 1.0.1, see comment by @zoya faberov.

  • Alex Putkov.1
    Alex Putkov.1 ✭✭✭✭✭

    Here you go

    n = 50
    df = pd.DataFrame()
    for ric_chunk in [ric_list[i:i + n] 
                      for i in range(0, len(ric_list), n)]:
        tmp_df, e = ek.get_data(ric_chunk,
                        ['TR.RNSFilerName',
                         'TR.RNSAnnouncedDate',
                         'TR.RNSTransactionType',
                         'TR.RNSARNumShrsTransacted',
                         'TR.RNSARPctOSTransacted',
                         'TR.RNSARTransactionPrice',
                         'TR.RNSARMktValTransaction',
                         'TR.RNSARTotShrsPostTrans',
                         'TR.RNSARPctOSPostTrans'])
        df = tmp_df.append(df)