Sentiment analysis on historical news for a portfolio

bd.sea

Dear all,
I am an early-stage user of the API environment, please excuse me for apparently redundant questions.

My aim is to gather management pay-related news from the last 365 days for every company in my portfolio (fewer than 550 RICs) at each fiscal year-end over the past 20 years, and perform sentiment analysis on each of them.

By reading the articles linked below, I have gathered part of what I need. However, I have some doubts related to the following:

1. Is this something that can be done, or are there limits? Consider that, when using the NEWS app, the MPAY news are not so many (per company, per year);
2. What’s the difference between using ek.get_news_headlines and rd.news.get_headlines? They seem to work in the same way (i.e., as shown in the figure in link 2), but I do not fully understand the actual differences;
3. Both links mention that portfolios can be used. However, due to my very basic knowledge of script structure, I did not understand how to implement this;
4. Both links specify that the time range can go up to 15 months. My doubt is: does this 15 months start from today (i.e., the last news that can be gathered was issued January 1st 2024) or from 15 months before the specified start date? When specifying an "old date" (e.g., 31-Dec-2022) in the script, I do not get any news for any company although in the NEWS app there are news on the topic: MPAY. Is this related to part of my service subscription, or is it a function issue?
5. Both links describe performing sentiment analysis, but I did not understand (at least on the 2nd link) whether it is done on the headline of the news or the full text. Apparently, it is performed on the headline field. Am I correct? Was that only for demonstration purposes? Would it be feasible to perform sentiment analysis on a longer and more complex text?
   

My script now is this:

from datetime import datetime, timedelta
import pandas as pd
import rd

start = datetime(2024, 12, 31).date()
end = start - timedelta(days=365)  
compNews = pd.DataFrame()
riclist = ['MSFT.O'.'AAPL.O']  

for ric in riclist:
    try:
        cHeadlines = rd.news.get_headlines("R:" + ric + ' AND Language:LEN AND Topic:MPAY', 
                                           start=str(start), 
                                           end=str(end), 
                                           count=100)
        cHeadlines['cRIC'] = ric
        if not compNews.empty:
            compNews = pd.concat([compNews, cHeadlines], ignore_index=True)
        else:
            compNews = cHeadlines
    except Exception as e:
        print(f"An error occurred with RIC {ric}: {e}")

compNews

Thank you for your time,

B

https://developers.lseg.com/en/article-catalog/article/introduction-news-sentiment-analysis-eikon-data-apis-python-example

https://developers.lseg.com/en/article-catalog/article/ai-driven-stock-insights--analyzing-financial-news-with-llms

Jirapongse

@bd.sea

Thank you for reaching out to us.

Please see the answers below.

1. Please check this Eikon Data API Usage and Limits Guideline. The current limit value (10-Oct-2019) is 100 headlines per request. The depth of history is 15 month.
2. The libraries were rebranded and redesigned. The first version is Eikon Data API (ek). The next one is Refinitiv Data Library and the latest one is LSEG Data Library for Python. We suggest to use the LSEG Data Library for Python. The examples are on GitHub. All libraries retrieve news from the same endpoint so the retrieved news should be the same.
3. You may contact your LSEG account team or sales team. The solution team may be able to implement this for you.
4. I ran the code to retrieve news for "R:LSEG.L". The latest news is on 2025-04-03 and the oldest news is on 2024-01-03.
5. The examples in the articles are for demonstration purposes. You can use them as starting points to implement your own applications. I checked the code and found that the Introduction to News Sentiment Analysis with Eikon Data APIs - a Python example uses stories while the AI-Driven Stock Insights:
   Analyzing Financial News with LLMs uses headlines.

bd.sea

https://community.developers.refinitiv.com/discussion/comment/125006#Comment_125006

Thank you so much for your time and suggestions,

only two more clarifications:

1. is there a chance that you could clarify how to modify the script so that it will collect the news for a portfolio in memory? By searching "portfolio" in the GitHub guides that you mentioned I have found Tutorials/Python Quant Tutorials/04. Portfolio Selection.ipynb but I did not get how to do that. Do I have to manually recreate the portfolio in the API? Or is there a way (and I am pretty sure there is) to recall a portfolio in memory with a specific name (e.g., MYPTF)?
2. Perhaps my previous point 4 was not clear. I was wondering whether you could access to historical "latest" news older than 15 months. Let's suppose for example that we want to gather the "latest" (max 100, of course) news for MSFT.O on a date between 01Jan2002 and 31dec2002. Can we do that? If yes, could you please explain how?

Thanks again,

B

Jirapongse

@bd.sea

You can access news headlines by using news queries.

For example, the news query of the code above is "R:PTT.BK". It returns news headlines that relate to PTT.BK.

You can construct your news query to retrieve news headlines that matches your requirements.

You can test the news query with the news monitor app.

Otherwise, you can contact the helpdesk team via MyAccount to create a news query for you. I am not sure if the news query works with Portfolio. You may contact the helpdesk team to verify it.

As far as I know, you can't access news older than 15 months via this API. We have other news products that can provide more historical news. Please contact your LSEG account team or sales team directly for more information.