Stale data from real-time RD retrieval

It might be easier if we're using the same code. It appears you may be running in VS Code. So that is where I'm doing all my testing. For example, try this entire example:

import refinitiv.data as rd
import os

print(f'lib version: {rd.__version__}')

os.environ["RD_LIB_CONFIG_PATH"] = "C:/USDLIBORCashFallback/"
rd.open_session()

stream = rd.open_pricing_stream(
    universe = ['USDSOFR='],
    fields = ['VALUE_DT1', 'PRIMACT_1']
)

prices = stream.get_snapshot()
print(prices)

rd.close_session()

I've created the same root directory as you and in there my config file looks like this:

{
  "logs": {
    "level": "debug",
    "transports": {
      "console": {
        "enabled": false
      },
      "file": {
        "enabled": true,
        "name": "/tmp/refinitiv-data-lib.log"
      }
    }
  },
  "sessions": {
    "default": "platform.prod",
    "platform": {
        "prod": {
            "app-key": "<your app key>",
            "username": "<machine id>",
            "password": "<password>",
            "signon_control": true
        }
    }   
  }
}

I've enabled the logs within the config file instead. When I run this in VS Code:

All comments

I performed the following test at approximately 21:40 UTC using version 1.6.0 of the Data Library for Python.

In the above, I don't see today's data (2024-05-06) but rather Friday's data (2024-05-03). If I pull the information using the SpeedGuide, I see the same values as above but do notice the TRADE_DATE field containing today's date.

Perhaps you can provide some screenshots to show the data you are retrieving. Also, I would suggest you update to the latest version of the Data Library for Python.

Hi @nick.zincone, thanks for response. Sorry I didn't describe the problem correctly... my app is looking for previous business date in the VALUE_DT1 field. Today it was getting back 2024-05-02 from get_data() call up until almost 12:22, although VALUE_DT1 had been updated to 2024-05-03 by 12:08.
I can't update immediately to latest version of RD library because it has breaking changes for other apps on my servers, I will need to find and make the modifications for all the apps at the same time.

The library should be utilizing the RDP Pricing Snapshot endpoint when making the call to rd.content.pricing.Definition().get_data(). And to confirm, when you are using the same call within both the rdp and rd libraries, you are getting different results. And when you use DataView it is consistent with the rdp libraries?

If you can confirm the behavior when using the RDP Pricing Snapshot endpoint, that will help the development team confirm the odd behavior. In addition, what would be helpful is if you can try the following code as well (assuming you have access to streaming content):

The snapshot call (get_data()) you are using should be getting values directly from the streaming services. So, if it is possible to try the above to confirm the values you are getting back from the streaming services that will help narrow down what is going on.

Hi @nick.zincone,

For some reason I am not able to succeed with code like above with get_stream() call, although that is much like the code we use with RDP realtime data request. Here is what the RDP code we use looks like, that is getting updated data for USDSOFR=/VALUE_DT1 updates in a timely manner:

When I try similar code with get_stream() call and RD lib, I don't get any error back but the code just hangs after the "Opening Stream" log message below, it never gets to the "Getting snapshot" log message, so it seems to be hung in the stream.open() call. Here is what that code looks like:

Since I can't seem to use RD get_stream() call, I tried get_data() instead which succeeds but the data is updated late.

Can you capture some logs within the rd library by doing the following before you open a session:

rd.get_config()["logs.transports.file.enabled"] = True
rd.get_config()["logs.transports.file.name"] = "data-lib.log"

This should create a log file that looks similar to this: 20240509-1922-776-data-lib.log (that is what was created on my machine).

thanks.

Hi @nick.zincone,

I don't seem to be able to find the log, I've tried adding the config directory to see if it would be put there but it doesn't help. It might not be saving the log because the app doesn't ever get to finish and I have to kill the window that's running it, or maybe I haven't added it right. I've tried above and below my code on lines 22-27:
Please let me know if I am not working in the log capturing right with the way I'm setting up the session.

It might be easier if we're using the same code. It appears you may be running in VS Code. So that is where I'm doing all my testing. For example, try this entire example:

import refinitiv.data as rd
import os

print(f'lib version: {rd.__version__}')

os.environ["RD_LIB_CONFIG_PATH"] = "C:/USDLIBORCashFallback/"
rd.open_session()

stream = rd.open_pricing_stream(
    universe = ['USDSOFR='],
    fields = ['VALUE_DT1', 'PRIMACT_1']
)

prices = stream.get_snapshot()
print(prices)

rd.close_session()

I've created the same root directory as you and in there my config file looks like this:

{
  "logs": {
    "level": "debug",
    "transports": {
      "console": {
        "enabled": false
      },
      "file": {
        "enabled": true,
        "name": "/tmp/refinitiv-data-lib.log"
      }
    }
  },
  "sessions": {
    "default": "platform.prod",
    "platform": {
        "prod": {
            "app-key": "<your app key>",
            "username": "<machine id>",
            "password": "<password>",
            "signon_control": true
        }
    }   
  }
}

I've enabled the logs within the config file instead. When I run this in VS Code:

Hi @nick.zincone, from log collected this morning it appears the date returned in VALUE_DT1 for USDSOFR= from get_data() call is updated as expected, so I need to look into why my app is not registering the update correctly.
I'm still a little concerned that I can't use the streaming snapshot if that is preferred by RD dev for snapshot data, so would appreciate if you can give any advice from looking at log included above.