Discover Refinitiv
MyRefinitiv Refinitiv Perspectives Careers
Created with Sketch.
All APIs Questions & Answers  Register |  Login
Ask a question
  • Questions
  • Tags
  • Badges
  • Unanswered
Search:
  • Home /
  • Eikon Data APIs /
avatar image
Question by bill39 · Sep 22, 2020 at 02:17 PM · api

A CSV download via the API scheduled every work day fails at least once per week with no apparent cause. How do I fix it?

#WARNING: the Eikon terminal must be running in order to connect to the API

#import packages
import eikon as ek # the Eikon Python wrapper package
import pandas as pd
import numpy as np
import datetime
from datetime import timedelta, date, datetime
from pandas.tseries.offsets import BDay
 
#connects to Bill's Eikon terminal
ek.set_app_key('XXXX')
 
#retreive the RICs from Eikon
df_rics,e = ek.get_data("lists('Inv Trust List')","TR.RIC")
 
#convert that into a list and set as an object
ric_list = df_rics['Instrument'].tolist()
 
#Slice, loop and concatenate the retrieval request - must be done in order to avoid a backend request timeout which
 #happens when we use too many RICs. n can be toggled below. 
n = 25
 df = pd.DataFrame()
for ric_chunk in [ric_list[i:i + n] 
                  for i in range(0, len(ric_list), n)]:
    tmp_df, e = ek.get_data(ric_chunk,
                    ['TR.RNSFilerName',
                     'TR.RNSAnnouncedDate',
                     'TR.RNSTransactionType',
                     'TR.RNSARNumShrsTransacted',
                     'TR.RNSARPctOSTransacted',
                     'TR.RNSARTransactionPrice',
                     'TR.RNSARMktValTransaction',
                     'TR.RNSARTotShrsPostTrans',
                     'TR.RNSARPctOSPostTrans'])
    df = tmp_df.append(df)
 
#set boundary dates of the download
end_date = date.today()
 start_date = end_date - BDay(2)
 
#set those dates in the necessary string format
end_date_str = datetime.strftime(end_date, "%Y-%m-%d")
 start_date_str = datetime.strftime(start_date, "%Y-%m-%d")
 
#denote dates in datetime format
df['RNS Announced Date'] = pd.to_datetime(df['RNS Announced Date'])
 
#set the date constraints as an object
mask = (df['RNS Announced Date'] >= start_date_str) & (df['RNS Announced Date'] < end_date_str)
 
#set the dataframe as only those with values inside the boundary dates
df = df.loc[mask]
 
#rename some of columns headings, some for compatibility purposes - some programs don't like the '£' symbol
df.rename(columns={'RNS Announced Date': 'Transaction Date','RNS AR Price (at Transaction) - £': 'RNS AR Price (at Transaction) GBP',
                    'RNS AR Market Value of Transaction - £': 'RNS AR Market Value of Transaction - GBP'},
           inplace=True)
 
#create file name and export as CSV
todays_date = date.today()
 todays_date_str = datetime.strftime(todays_date, "%Y%m%d")
 df.to_csv('Daily API Download_' + todays_date_str + '.csv')

People who like this

0 Show 5
Comment
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
bill39 · Sep 22, 2020 at 02:21 PM 0
Share

This is the error


 
                  

Error code 503 | Server Error: API Proxy is not available

Traceback (most recent call last):

File "daily_api_download.py", line 15, in <module>

df_rics,e = ek.get_data("lists('Inv Trust List')","TR.RIC")

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\data_grid.py", line 186, in get_data

result = eikon.json_requests.send_json_request(_endpoint, payload, debug=debug)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 121, in send_json_request

_raise_for_status(response)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 244, in _raise_for_status

raise EikonError(response.status_code, error_msg)

eikon.eikonError.EikonError: Error code 503 | Server Error: API Proxy is not available

avatar image
bill39 · Oct 20, 2020 at 04:21 PM 0
Share

UPDATE

Hi @zoya.farberov, as explained on another post, I have been logging my errors and there are three scenarios that occur:

1) download works (happens c.25% of the time),

2) download fails because Eikon Terminal is not open at the time of running (happens c.25% of the time) -this is a Windows Task Scheduling error and not one I need help fixing,

3) download fails and I get the error code attached (happens 50% of the time)

Please would you help me resolve scenario 3. The error code that is generated is attached. ThanksError log.zip

error-log.zip (660 B)
avatar image
bill39 bill39 · Oct 22, 2020 at 09:25 AM 0
Share

Hi @zoya.farberov , please would you get back to me on this? Many thanks

avatar image
REFINITIV
zoya.farberov ♦♦ bill39 · Oct 22, 2020 at 09:02 PM 0
Share

Hi @pierre.faurel,

Are you able to take a look at the follow-up information on the issue and the logs from @bill39?

Thanks


3 Replies

  • Sort: 
avatar image
REFINITIV
Best Answer
Answer by pierre.faurel · Nov 12, 2020 at 10:06 AM

Hi @bill39

The two fixes I proposed are workaround to detect errors and manage it when it's possible (that's what I named "secure your code").

1) Check that session is started and available with this code:

cnx_state = ek.get_desktop_session().get_open_state()
if cnx_state != ek.Session.State.Open:
     exit(-1)

If the cnx_state is equal to ek.Session.State.Closed, that means Eikon API Proxy is not avalaible (need to restart Eikon ?).

2) Catch 2504 error code to request again

    while retry:
        try:
            tmp_df, e = ek.get_data(ric_chunk,                             ['TR.RNSFilerName',
                ... 
               'TR.RNSARPctOSPostTrans'])
            retry = False
        except EikonError as e:
            if e.code not in [2504]:
                retry = False
            else:
                time.sleep(0.5) # delay retry in 500 msec

Note that you can add other error code than 2504 in the list to catch (ex: 400).

Comment
wasin.waeosri

People who like this

1 Show 1 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
bill39 · Dec 07, 2020 at 12:24 PM 0
Share

Hi @pierre.faurel, please would you help me integrate this into my code? Firstly for part 1), I am getting

AttributeError: module 'eikon' has no attribute 'get_desktop_session'

and secondly I have already incorporated an error catching code to stop error code 400 (which I don't understand either) following guidance from one of your colleagues. I don't know how I should be incorporating the two together, unfortunately now this has gotten beyond my skill level - I'm a stock analyst by trade, not a programmer! Please help. Many thanks. Ideally we would set up a remote session on my computer where we can go through it and amend the code as necessary.

avatar image
REFINITIV
Answer by zoya.farberov · Sep 22, 2020 at 02:59 PM

Hello @bill39,

I understand that the cause of the issue is always error 503? And you seem to be seeing the error consistently, as opposite to once or twice? If this is correct, this is not quite the expected behavior.

Please see this relevant previous discussion, the Eikon troubleshooting guide is very often a key tool to determine the specific cause for the issue, hopefully, helpful to you.

Some other relevant factors are: are you on the recent version of Eikon? Are you on the latest version of the Eikon library, as some important fixes have been released in version of 1.1.6.post2. The issues fixed also appear to be tracked to python 3.8+, if you are on python 3.8+, the use of 1.1.6.post2 is recommended.

Hope that this info is of help.



Comment

People who like this

0 Show 5 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
bill39 · Sep 25, 2020 at 10:01 AM 0
Share
Yes 503 is always the error. I have python version 3.8.+. For some reason the troubleshooting guide link you have provided directs me to a page that 'cannot be found'. Please would you email me the troubleshooting instructions for error code 503? Thanks
avatar image
REFINITIV
zoya.farberov ♦♦ bill39 · Sep 25, 2020 at 03:08 PM 0
Share

Hello @bill39,

Please try again- the article link should work now.

The dev portal went through a major update today and some links were migrated.

Thanks for letting us know

avatar image
bill39 · Sep 28, 2020 at 11:10 AM 0
Share

The API request ran today when I executed it manually but it didn't run when run by my Task Scheduler early this morning. What could be the cause of the requests sometimes working and sometimes not? And how do I fix it? Thank you

avatar image
REFINITIV
zoya.farberov ♦♦ bill39 · Sep 29, 2020 at 04:26 PM 0
Share

Hello @bill39,

Does it run via scheduler in general for you? If you schedule it in 1 min, does it run?

Eikon/Refinitiv Workspace desktop needs to be up, running and logged in, Eikon Proxy within it up and running as part of it, for Eikon Data API python to interact with it. If that is not the case, the task will not run successfully. By default, Eikon desktop is logged out at night, do you find yourself logging back in, in the morning?

avatar image
bill39 zoya.farberov ♦♦ · Oct 02, 2020 at 09:36 AM 0
Share

If I schedule it in 1 minute it does sometimes work but not always. I.e. some days it will run every time I try and other days it does not. I have also set a task to open the Eikon terminal every morning before the API download so it is always on when the API runs, yes. I also run a macro in an excel file every morning that uses Eikon data and sometimes this fails also. When this macro fails the buyback also fails but there are times when the macro runs and the API download fails.

avatar image
REFINITIV
Answer by zoya.farberov · Oct 02, 2020 at 03:47 PM

Hello @bill39,

As you mention that, for the most part, the scheduled task succeeds, and sometimes it does not, I think it would be helpful to collect any error or info at that time that it fails. Try introducing

import eikon as ek
...
import logging.config 
ek.set_log_level(logging.DEBUG)
ek.set_app_key(XXXXXX

please collect and share any info that is logged at the time of the failed call?

Let's see if we can find out more to help us track down the issue.

---

Would like to also note, that the resulting CSV content is subject to Eikon, Eikon Excel and any data obtained via Eikon Data API single user license, to be used by logged in Eikon user only. While sharing of the content would require Enterprise level product and license.


Comment

People who like this

0 Show 4 · Share
10 |1500 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

avatar image
bill39 · Oct 05, 2020 at 09:49 AM 0
Share

Noted, thanks. Thanks, I have added the logging lines to my code. Where will the error log be stored? When running the script manually today the error I receive is:


Traceback (most recent call last):

File "daily_api_download.py", line 39, in <module>

'TR.RNSARPctOSPostTrans'])

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\data_grid.py", line 186, in get_data

result = eikon.json_requests.send_json_request(_endpoint, payload, debug=debug)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 118, in send_json_request

_check_server_error(result)

File "C:\Users\william\AppData\Local\Continuum\anaconda3\lib\site-packages\eikon\json_requests.py", line 194, in _check_server_error

raise EikonError(int(server_response['ErrorCode']), error_message)

eikon.eikonError.EikonError: Error code 2504 | UDF Core request failed. Gateway Time-out

_______________________________________


Thanks

avatar image
REFINITIV
pierre.faurel bill39 · Oct 05, 2020 at 10:36 AM 0
Share

Hi,

error code 2504 is something you can catch to retry to send your request.

You can update your code as below:

import time

...

for ric_chunk in [ric_list[i:i + n]
    retry = True
    while retry:
        try:
            tmp_df, e = ek.get_data(ric_chunk,
                            ['TR.RNSFilerName',
                             'TR.RNSAnnouncedDate',
                             'TR.RNSTransactionType',
                             'TR.RNSARNumShrsTransacted',
                             'TR.RNSARPctOSTransacted',
                             'TR.RNSARTransactionPrice',
                             'TR.RNSARMktValTransaction',
                             'TR.RNSARTotShrsPostTrans',
                             'TR.RNSARPctOSPostTrans'])
            retry = False
        except EikonError as e:
            if e.code not in [2504]:
                retry = False
            else:
                time.sleep(0.5) # delay retry in 500 msec
    df = tmp_df.append(df)

For the Error code 503 | Server Error: API Proxy is not available, if you continue to reproduce (a fix with eikon 1.1.6.post3 was published on pypi), as you trigger your script to run daily, that's possible that Eikon Proxy turned down.

You can secure your script with :

ek.set_app_key('XXXX')

cnx_state = ek.get_desktop_session().get_open_state()
if cnx_state != ek.Session.State.Open:
    exit(-1)


avatar image
bill39 pierre.faurel · Oct 06, 2020 at 04:42 PM 0
Share

How would you suggest I integrate that into my current code? It already splits my retreival into chunks like this:


n = 10
 df = pd.DataFrame()
for ric_chunk in [ric_list[i:i + n] 
                  for i in range(0, len(ric_list), n)]:
    tmp_df, e = ek.get_data(ric_chunk,
                    ['TR.RNSFilerName',
                     'TR.RNSAnnouncedDate',
                     'TR.RNSTransactionType',
                     'TR.RNSARNumShrsTransacted',
                     'TR.RNSARPctOSTransacted',
                     'TR.RNSARTransactionPrice',
                     'TR.RNSARMktValTransaction',
                     'TR.RNSARTotShrsPostTrans',
                     'TR.RNSARPctOSPostTrans'])
    df = tmp_df.append(df)

Additionally, what do you mean by "securing my script"? What does that code do?

Thank you

Watch this question

Add to watch list
Add to your watch list to receive emailed updates for this question. Too many emails? Change your settings >
13 People are following this question.

Related Questions

HTTP or PYTHON APIs

Carbon time series API (R)

Why is my Jupyter Notebook not executing my eikon API code?

eikon data api bulk data

Equity fund ownership

  • Feedback
  • Copyright
  • Cookie Policy
  • Privacy Statement
  • Terms of Use
  • Careers
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Alpha
  • App Studio
  • Block Chain
  • Bot Platform
  • Calais
  • Connected Risk APIs
  • DSS
  • Data Fusion
  • Data Model Discovery
  • Datastream
  • Eikon COM
  • Eikon Data APIs
  • Elektron
    • EMA
    • ETA
    • WebSocket API
  • Legal One
  • Messenger Bot
  • Messenger Side by Side
  • ONESOURCE
    • Indirect Tax
  • Open PermID
    • Entity Search
  • Org ID
  • PAM
    • PAM - Logging
  • ProView
  • ProView Internal
  • Product Insight
  • Project Tracking
  • Refinitiv Data Platform
    • Refinitiv Data Platform Libraries
  • Rose's Space
  • Screening
    • Qual-ID API
    • Screening Deployed
    • Screening Online
    • World-Check One
    • World-Check One Zero Footprint
  • Side by Side Integration API
  • TR Knowledge Graph
  • TREP APIs
    • CAT
    • DACS Station
    • Open DACS
    • RFA
    • UPA
  • TREP Infrastructure
  • TRIT
  • TRKD
  • TRTH
  • Thomson One Smart
  • Transactions
    • REDI API
  • Velocity Analytics
  • Wealth Management Web Services
  • World-Check Data File
  • Explore
  • Tags
  • Questions
  • Badges