Incorrect pandas dependency in python eikon data api

Options
davidk
davidk Contributor
in Eikon Data APIs

The definitions of eikon versions 1.1.6, 1.1.7, and 1.1.8 on pip and conda-forge indicate that these versions depend on pandas >= 0.17.0. However, the source of these versions reference the DataFrame.convert_dtypes method, which was only added in 1.0.0. Thus the code is only compatible with pandas >= 1.0.0 and even the most basic calls (e.g. eikon.get_data('AAPL.O', 'TR.RIC') ) fail with pandas < 1.0.0.


Please fix these dependency definitions to indicate the dependence on 1.0.0, or conditionally code to support the listed versions. Supporting the listed versions would of course be preferable since the versioning indicates these are bugfix releases. Unfortunately pandas has changed this API several times... to truly support 0.17.0+, the correct code should be something like this:


if pd.__version__ < '0.20.0':

df = df.convert_objects()

elif pd.__version__ < '1.0.0':

df = df.infer_objects()

else:

df = df.convert_dtypes()



Tagged:

Best Answer

  • pf
    pf LSEG
    Answer ✓

    Hi @davidk,

    Unfortunately, we won't manage backward compatibility with previous versions of pandas (or other dependency).

    But if we look the issue from your point of view, you can apply a fix on your side to be forward compatible with latest eikon version.
    Install eikon 1.1.8 then downgrade pandas to the version you need (0.24.2).
    Then, you can add following lines in your script/project:

    import panda as pd
    if pd.__version__ < '0.20.0':
        pd.DataFrame.convert_dtypes = pd.DataFrame.convert_objects
    elif pd.__version__ < '1.0.0':
        pd.DataFrame.convert_dtypes = pd.DataFrame.infer_objects

    With this fix, you should be able to use pandas < 1.0.0 (and you'll remove it as soon as you'll be able to upgrade to 1.x.x).
    On my side , I tested successfully to downgrade pandas to 0.24.2 then run a script with this fix.

Answers

  • pf
    pf LSEG

    Hi,

    You're right. This was detected and fixed with the latest version 1.1.8 of eikon lib.

    From now, it needs 1.0.0 pandas version.


  • davidk
    davidk Contributor
    https://community.developers.refinitiv.com/discussion/comment/70057#Comment_70057

    I see 1.1.8 listed as pandas >= 0.17.0 on conda-forge? At least that's what comes up when I do a


    conda search -c conda-forge eikon==1.1.8 --info


    Also though I think it would be advisable to find some way to prevent people with < 1.0.0 pandas from getting 1.1.5+ going forward, if that's possible. I don't know how common it is industry-wide, but at least in our firm's case we're stuck on pandas 0.24.2 for the near future-- we have a legacy code base where we're still trying to extricate pandas.Panel, and we also heavily use PySpark (which is dependent on pandas 0.24.2).

  • davidk
    davidk Contributor
    https://community.developers.refinitiv.com/discussion/comment/70057#Comment_70057

    @pierre.faurel This just got more serious.


    Eikon 4.0.53 was just pushed to us yesterday, and it breaks the compatibility of pyeikon 1.1.2 with Eikon. This is because the .portInUse file was moved from the ThomsonReuters-based folder scheme to the Refinitiv-based folder scheme supported in 1.1.2. Earlier versions of Eikon appeared to have been forwarding requests on the old port to the new port, but 4.0.53 does not.


    As a result, pyeikon 1.1.2 no longer works with Eikon. At the same time, pyeikon > 1.1.2 no longer works with pandas < 1.0.0.


    As a result, there now exists NO WORKING VERSION of the Eikon Data API for Python that is compatible with pandas < 1.0.0. Again, not all other packages yet support pandas 1.0.0+ and I am quite sure we're not the only firm still working on our upgrade path. For now our firm has had to roll back to Eikon 4.0.52.


    If necessary we could vendor pyeikon 1.1.8 and hack it to remove the offending convert_dtypes call but this is obviously inferior to a supported solution (there also may be other calls we have not hit yet).

  • davidk
    davidk Contributor
    https://community.developers.refinitiv.com/discussion/comment/70610#Comment_70610

    Thanks Pierre. Monkey patching pandas is of course slightly frowned upon but it does appear to work with your use case since you're not using the kwargs to convert_dtypes. I'm going to continue testing in case there are issues I haven't hit, but at least this gets me to the point where eikon.get_data('AAPL.O', 'TR.Ric') works successfully on our standard environment.


    I now see eikon 1.1.8 defined as requiring >= 1.0.0 on conda-forge which is good. As pandas is now using semantic versioning hopefully these API breaks should become rarer, but it might be wise in the future to define your version restriction as pandas >= 1.0.0, pandas < 2.0 since there will be API breaks in 2.0 relative to 1.x (per the semver contract). However, code written to pandas 1.0.0 should still function on the whole 1.x line.

Categories