For a deeper look into our Eikon Data API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials |  Articles

question

Upvotes
Accepted
davidk avatar image
244 ●10 ●11 ●16

Incorrect pandas dependency in python eikon data api

The definitions of eikon versions 1.1.6, 1.1.7, and 1.1.8 on pip and conda-forge indicate that these versions depend on pandas >= 0.17.0. However, the source of these versions reference the DataFrame.convert_dtypes method, which was only added in 1.0.0. Thus the code is only compatible with pandas >= 1.0.0 and even the most basic calls (e.g. eikon.get_data('AAPL.O', 'TR.RIC') ) fail with pandas < 1.0.0.


Please fix these dependency definitions to indicate the dependence on 1.0.0, or conditionally code to support the listed versions. Supporting the listed versions would of course be preferable since the versioning indicates these are bugfix releases. Unfortunately pandas has changed this API several times... to truly support 0.17.0+, the correct code should be something like this:


if pd.__version__ < '0.20.0':

df = df.convert_objects()

elif pd.__version__ < '1.0.0':

df = df.infer_objects()

else:

df = df.convert_dtypes()



eikoneikon-data-apipythonrefinitiv-dataplatform-eikonworkspaceworkspace-data-apierrordependency
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

2 Answers

Upvotes
Accepted
pierre.faurel avatar image
4.3k ●2 ●4 ●5

Hi @davidk,

Unfortunately, we won't manage backward compatibility with previous versions of pandas (or other dependency).

But if we look the issue from your point of view, you can apply a fix on your side to be forward compatible with latest eikon version.
Install eikon 1.1.8 then downgrade pandas to the version you need (0.24.2).
Then, you can add following lines in your script/project:

import panda as pd
if pd.__version__ < '0.20.0':
    pd.DataFrame.convert_dtypes = pd.DataFrame.convert_objects
elif pd.__version__ < '1.0.0':
    pd.DataFrame.convert_dtypes = pd.DataFrame.infer_objects

With this fix, you should be able to use pandas < 1.0.0 (and you'll remove it as soon as you'll be able to upgrade to 1.x.x).
On my side , I tested successfully to downgrade pandas to 0.24.2 then run a script with this fix.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

davidk avatar image davidk

Thanks Pierre. Monkey patching pandas is of course slightly frowned upon but it does appear to work with your use case since you're not using the kwargs to convert_dtypes. I'm going to continue testing in case there are issues I haven't hit, but at least this gets me to the point where eikon.get_data('AAPL.O', 'TR.Ric') works successfully on our standard environment.


I now see eikon 1.1.8 defined as requiring >= 1.0.0 on conda-forge which is good. As pandas is now using semantic versioning hopefully these API breaks should become rarer, but it might be wise in the future to define your version restriction as pandas >= 1.0.0, pandas < 2.0 since there will be API breaks in 2.0 relative to 1.x (per the semver contract). However, code written to pandas 1.0.0 should still function on the whole 1.x line.

Upvotes
pierre.faurel avatar image
4.3k ●2 ●4 ●5

Hi,

You're right. This was detected and fixed with the latest version 1.1.8 of eikon lib.

From now, it needs 1.0.0 pandas version.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

davidk avatar image davidk

I see 1.1.8 listed as pandas >= 0.17.0 on conda-forge? At least that's what comes up when I do a


conda search -c conda-forge eikon==1.1.8 --info


Also though I think it would be advisable to find some way to prevent people with < 1.0.0 pandas from getting 1.1.5+ going forward, if that's possible. I don't know how common it is industry-wide, but at least in our firm's case we're stuck on pandas 0.24.2 for the near future-- we have a legacy code base where we're still trying to extricate pandas.Panel, and we also heavily use PySpark (which is dependent on pandas 0.24.2).

davidk avatar image davidk

@pierre.faurel This just got more serious.


Eikon 4.0.53 was just pushed to us yesterday, and it breaks the compatibility of pyeikon 1.1.2 with Eikon. This is because the .portInUse file was moved from the ThomsonReuters-based folder scheme to the Refinitiv-based folder scheme supported in 1.1.2. Earlier versions of Eikon appeared to have been forwarding requests on the old port to the new port, but 4.0.53 does not.


As a result, pyeikon 1.1.2 no longer works with Eikon. At the same time, pyeikon > 1.1.2 no longer works with pandas < 1.0.0.


As a result, there now exists NO WORKING VERSION of the Eikon Data API for Python that is compatible with pandas < 1.0.0. Again, not all other packages yet support pandas 1.0.0+ and I am quite sure we're not the only firm still working on our upgrade path. For now our firm has had to roll back to Eikon 4.0.52.


If necessary we could vendor pyeikon 1.1.8 and hack it to remove the offending convert_dtypes call but this is obviously inferior to a supported solution (there also may be other calls we have not hit yet).