Please can you explain why there is a lag on receiving EIA stats data via a script i have written?

Terence_97

Hi team, I am posting on behalf of an external client for the query below, please help and prioritize:

Please can you explain why there is a lag on receiving EIA stats data via a script i have written using the appkey?

from lseg.data.content import historical_pricing as hp
from datetime import date
import os

1. Correct RIC for total commercial crude stocks

crude_rics = ["C-STK-T-EIA"]

2. Date range

start_date = "2020-01-01"
end_date = str(date.today())

3. Fetch data

response = hp.summaries.Definition(
universe=crude_rics,
interval=hp.Intervals.WEEKLY,
start=start_date,
end=end_date
).get_data()

4. Extract and rename

raw_df = response.data.df
print("Latest date returned:", raw_df.index.max())
print("Today's date:", date.today())

comm_last_df = raw_df[["COMM_LAST"]].copy()
comm_last_df.columns = ["TotalCrude"]

5. Save

os.makedirs("../data", exist_ok=True)
comm_last_df.to_csv("../data/crude_total_stocks.csv")
print("Saved:", os.path.exists("../data/crude_total_stocks.csv"))

I can confirm that there should be values for 1st of August but it seems that the script above pulls a week late.

Jirapongse

@Terence_97

Thank you for reaching out to us.

Currently, I could retrieve the data properly.

The library retrieves data from the Historical Pricing API. You can verify this issue by running the following formula in the LSEG Workspace Excel or contacting the Historical Pricing API support team directly when the problem occurred.

=@RDP .HistoricalPricing("C-STK-T-EIA","COMM_LAST","START:01-Jan-2020 END:07-Aug-2025 INTERVAL:P1W SOURCE:RFV",,"CH:Fd RH:Timestamp",B2)

Terence_97

https://community.developers.refinitiv.com/discussion/comment/127307#Comment_127307

Thanks for the response. I can see on excel as well that this is working fine. But it looks like the script of the client doesn't pull the same dates.

I have checked the entitlements and it seems good.

Jirapongse

Currently, the API works fine too.

This issue is similar to the case (14802400).

Terence_97

Here's what the client had mentioned:

"Thanks for your help. My script worked by 17.20 when I checked again. The issue was that it didn't work at 15.30 (UK time) which is when I need it, so that I have live information for analysis.

So if there is a fault in my script that means there is a lag from 15.30 to 17.20 that you can spot that means I am not getting it instantaneously that would be great. Or is this due to my permissions? For context using a similar script and trying from the EIA website directly the week before also presented with a lag. Is that a nuance of using APIs?"

Jirapongse

@Terence_97

You need to check with the Historical Pricing API team when the data was released.