Dear Laurens,

I am the product manager for the Eikon Library and Refinitiv Data Libraries. Apologies for jumping into this conversation that late and – as you pointed out – for the communication that could have been handled better. This has created some confusion which I would like to clarify. Hopefully, this will give you better visibility into the upcoming changes and the potential impact on your code and your applications.

First, I would like to explain our strategy for the Eikon library and Refinitiv Data Libraries:

• We continue to support and maintain the Eikon Library
  This library is "Feature Complete", but we continue to support it and provide maintenance releases.
• The Refinitiv Data Library will be renamed the LSEG Data Library (LD Lib)
  Due to LSEG's acquisition of Refinitiv, Refinitiv products and libraries are rebranded as LSEG. Rebranding a library means changing its name, which is a breaking change. This change will occur in version 2.0 of our data library.
• We are taking advantage of the name change of our data library to slightly improve its public interfaces
  Some aspects of the public APIs will be slightly improved in the LSEG Data Library 2.0. The “eikon” module which is a duplicate of the Eikon library will be removed.
• We will continue to support and maintain the Refinitiv 1.x data library
  After the rebranding and release of LSEG Data Library 2.0, the Refinitiv Data Library will become " Feature Complete", but we will continue to support it and provide maintenance releases. This is essential for customers who cannot migrate to the LSEG Data Library 2.0. However, new features will only be added to the LSEG data library.
• The “eikon” module will not be removed from the Refinitiv 1.x data library
  The FutureWarning you noticed was about the change to the LSEG Data Library 2.0. Maintenance releases of RD Lib will continue to include the “eikon” module. There are no plans to remove it.
• We will ensure that there are no dependency conflicts between the Eikon library, the Refinitiv data library and the LSEG data library
  This is essential for CodeBook and many of our customers. This check will be integrated into our automated tests.
• The underlying data sources used by the LSEG Data Library's get_data() function will be fully aligned with those used by the Eikon Library's ek.get_data()
  This will ensure a smooth migration from the Eikon library to the LSEG Data Library.

In other words, you can continue to use the Refinitiv Data Library as you do today. New versions of this library will continue to include the “eikon” module. Whenever and wherever you want, you can migrate to the LSEG Data Library at your own pace. You will be able to use it in the same environment as the Eikon Library and the Refinitiv Data Library

I hope this information provides you with more clarity. My understanding is that these changes will not impact your code and will give you the flexibility to migrate to the LD Lib whenever you want and need to.

Hello, Laurens.

Thank you for your feedback.

Future v2 version (2.0.0) is a major version that allow to have breaking changes. If any breaking changes will we have - we should introduce major changing version (https://semver.org/).

This is a normal practice to introduce breaking changes in product.

That’s why library have FutureWarnings in advance to let users know that changes will take place.

If you are currently use Eikon, you can still use it as separate package.

Regarding release news.

As soon we released new version, immediately we provide release notes for this version - https://developers.lseg.com/en/api-catalog/refinitiv-data-platform/refinitiv-data-library-for-python/documentation#release-notes

I ran you code example. Please take a look results below for 1.5.0 version. I attached the log file.

apple.req.log.zip

Dear Andrii,

1. Regarding deprecation.

Suppose you run a railway network and you suddenly decide to change the track width. You can imagine that all your customers that have trains who can only run on the old track will not be happy that they can not run their perfectly fine trains on the track anymore.

As stated before you always need a combination of RD and Eikon (let's say e.g. that you have perform a RD search command and then get an eikon timeseries command). Therefore you need both packages. Both packages can not be reliably used together due to conflicting dependencies (which is already currently the case) between the existing RD and Eikon python packages.

I don't want to get in a discussion regarding versioning semantics. The fact that something can be done doesn't mean that something should be done.

2. Regarding rd_GetData()

I am highly surprised you get the answer as stated above. Both in my own python environment and in Eikon CodeBook i am not able to reproduce this...

I checked what version codebook was running (this was 1.2.0) so that could explain something. However: Refinitiv is responsible for Codebook and this command is clearly not returning what it should be returning

Then I ran again the 1.5.0 code in my own environment and also could not reproduce your results (and please find my log attached):

import refinitiv.data as rd

rd.__version__

rd.open_session()missingfields.txt

fields= ["CF_CURR", "CF_EXCHNG", "TR.PriceClose.Currency", "TR.InstrumentType", "TR.ExchangeName"]

EikonData = rd.eikon.get_data(instruments= "AAPL.O", fields= fields)

print(EikonData)

RD_Data = rd.get_data(universe="AAPL.O", fields= fields)

print(RD_Data)

print(RD_Data.columns)

3. Regarding communication

let me ask 3 questions:

- Have users been asked if eikon could be removed and when?

- Has Refinitiv ever actively reached out to users about these changes? Wouldn't one expect a news letter that contains some sort of roadmap.

- if I pay thousands of pounds per month to use this service would one expect this deprecations?

4. Conclusion

I would like conclude that it is not normal practice that the future legacy function receives data and then it suddenly disappears when I use the next version of that function.

Furthermore active communication reaching users about product planning is lacking. Every week I get emails from a nice training manager about how I can do something in excel. Not a single time i receive an email regarding the api which all my workflows use...

Furthermore it is extremely disappointing that no questions are asked to users about why they pay for the product and what they expect from the product...

I don't use this product because I like programming. I use this product to earn money.

Laurens, thank you for the feedback.

Regarding rd.get_data().

As I see at your log file you have the permission issue. You've got a server response message:

{
  "ID": 5,
  "Type": "Status",
  "Key": {
    "Service": "IDN_FD3",
    "Name": "AAPL.O"
  },
  "State": {
    "Stream": "Closed",
    "Data": "Suspect",
    "Code": "NotEntitled",
    "Text": "Access Denied: User req to PE(74)"
  }
}

More generally, permissions (list of PEs) give access to specific markets (by region and/or by domain). PE (Permission Entitlement) give access to list of instruments.

I would recommend contact your Refinitiv Account team or Sales team directly to verify the permission to the requested data.

perfectOMMstream.txtDear Andrii,

Thank you for your reply, however I am bit disappointed that you only looked at the last lines of the logfile and did not try to identify the root cause of the difference between rd.eikon.get_data() and rd.get_data()

I was extremely surprised (again) that in rd.eikon.get_data() I don't have a permission error and that I suddenly have a permission error for a live data stream [OMM] in rd.get_data() while these fields ["CF_CURR", "CF_EXCHNG"] are simply static character fields that do not have much in common with live data as they won't change in time.

So I checked the previously supplied log (missingfields.txt) in detail which api (json) calls are made:

1. rd.eikon.get_data() call

HTTP Request id 2
url = http://localhost:9060/api/udf
method = POST
headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJBcGlWZXJzaW9uIjoiMSIsIkFwcEtleSI6IjEyMDU0YjRlN2JiMTRlZTU4YWM2MzQ3MjQ4ZDMzNzg2MGY1MmUxZDUiLCJBcHBTY29wZSI6InRyYXBpIiwiTGlicmFyeU5hbWUiOiJSRFAgUHl0aG9uIExpYnJhcnkiLCJMaWJyYXJ5VmVyc2lvbiI6IjEuNS4wIiwiaWF0IjoxNzAxMTIxMDEyLCJleHAiOjE3MDIzMzA2MTJ9.RobPtR-hlTdO_pDm-MbVrrq2h7lTSdQ-qr7XnLZyRis', 'x-tr-applicationid': 'DEFAULT_WORKSPACE_APP_KEY'}
json = {'Entity': {'E': 'DataGrid_StandardAsync', 'W': {'requests': [{'instruments': ['AAPL.O'], 'fields': [{'name': 'CF_CURR'}, {'name': 'CF_EXCHNG'}, {'name': 'TR.PriceClose.Currency'}, {'name': 'TR.InstrumentType'}, {'name': 'TR.ExchangeName'}]}]}}, 'ID': '123'}

as you can see clearly here udf is called for ["CF_CURR", "CF_EXCHNG"]

fields': [{'name': 'CF_CURR'}, {'name': 'CF_EXCHNG'}

is in the body of this json request

2. When I look then in rd.get_data()

HTTP Request id 3
url = http://localhost:9060/api/udf
headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJBcGlWZXJzaW9uIjoiMSIsIkFwcEtleSI6IjEyMDU0YjRlN2JiMTRlZTU4YWM2MzQ3MjQ4ZDMzNzg2MGY1MmUxZDUiLCJBcHBTY29wZSI6InRyYXBpIiwiTGlicmFyeU5hbWUiOiJSRFAgUHl0aG9uIExpYnJhcnkiLCJMaWJyYXJ5VmVyc2lvbiI6IjEuNS4wIiwiaWF0IjoxNzAxMTIxMDEyLCJleHAiOjE3MDIzMzA2MTJ9.RobPtR-hlTdO_pDm-MbVrrq2h7lTSdQ-qr7XnLZyRis', 'x-tr-applicationid': 'DEFAULT_WORKSPACE_APP_KEY'}
json = {'Entity': {'E': 'DataGrid_StandardAsync', 'W': {'requests': [{'instruments': ['AAPL.O'], 'fields': [{'name': 'TR.PRICECLOSE.CURRENCY'}, {'name': 'TR.INSTRUMENTTYPE'}, {'name': 'TR.EXCHANGENAME'}]}]}}}

I can then clearly see that here the fields are missing from the SAME SERVICE. the fields parameters here omits ["CF_CURR", "CF_EXCHNG"]

'fields': [{'name': 'TR.PRICECLOSE.CURRENCY'}

3. OMM stream.

It then tries to get these fields from the OMM stream. It is indeed not able to find this fields in this stream. Off course as proved above these fields have nothing to do with a live streaming. For the avoidance of any doubt I decided to set up a simple OMM stream to see if I really did have a permission issue with OMM streaming.

import refinitiv.data as rd
from refinitiv.data.delivery import omm_stream
import datetime
import json

rd.open_session()

# Callback function to display data or status events
def display_event(eventType, event):
    currentTime = datetime.datetime.now().time()
    print("----------------------------------------------------------")
    print(">>> {} event received at {}".format(eventType, currentTime))
    print(json.dumps(event, indent=2))
    return

stream = omm_stream.Definition(
    name="EUR=",
    fields=['BID', 'ASK','OPEN_PRC','HST_CLOSE','TIMACT'],
    domain='MarketPrice').get_stream()
# Update - as and when field values change, we receive updates from the server
stream.on_update(lambda event, item_stream : display_event("Update", event))

stream.open()

I checked this and the stream ran perfectly fine. (see also log perfectOMMstream.txt)

perfectOMMstream.txt

4. Conclusion

This is as proven above definitely not a permission problem. This is a bug in rd.get_data() that it asks the wrong fields to the OMM service. Furthermore it is bad design of the rd.get_data() function as the old rd.eikon.get_data() function can do it in a single api call. The new function rd.get_data() requires an additional call to the OMM service and fails because it can not find this field in the service and blames it on some permission. It is however very likely that these static fields are not even available in this OMM service!

So for the third time I have extensively proven that this is a real and serious issue. PLEASE fix this!

Dear, Laurens.

I see that you disappointed.

The design of rd.get_data() gives User ability request fundamental fields data from UDF or RDP platform and pricing fields data. Different endpoints provides these data, that's why we have two requests accordingly.

I have assumption, that the issue you have causes by permission lack. I ask you kindly double check with next code examples:

Regarding OMMstream.

import refinitiv.data as rd
from refinitiv.data.delivery import omm_stream
import datetime
import json

rd.open_session()


def display_event(eventType, event):
    currentTime = datetime.datetime.now().time()
    print("----------------------------------------------------------")
    print(">>> {} event received at {}".format(eventType, currentTime))
    print(json.dumps(event, indent=2))


stream = omm_stream.Definition(
    name="AAPL.O", fields=["CF_CURR", "CF_EXCHNG", "DSPLY_NAME"], domain="MarketPrice"
).get_stream()
stream.on_refresh(lambda event, item_stream: display_event("Refresh", event))

stream.open(with_updates=False)

as you can see we use the same Instrument with the same fields from initial message. Please provide the log file for this run.

Regarding rd.get_data() I ran the same code for RD library 1.5.0 from initial message.

import refinitiv.data as rd

rd.open_session()

fields = ["CF_CURR", "CF_EXCHNG", "TR.PriceClose.Currency", "TR.InstrumentType", "TR.ExchangeName", "DSPLY_NAME"]

df = rd.get_data(universe="AAPL.O", fields=fields)
print(df.to_string())

I attached the log file.

rd.get_data.1.5.0.log.zip