Adding story text to output using Python API

s.tank

Hi

I currently have the following code that is fetching specific new articles using Python API. However I am not sure how to add the story text to the output. I currently have the Headline, StoryId, and Link to the article.

Code:

df = ek.get_news_headlines(
'Topic:GB AND (SOURCE:CMPNY OR SOURCE:MCE OR SOURCE:GEN)',
date_from='2025-07-23',
date_to='2025-07-30')
df

df['Links']=""
for idx, story in enumerate(df['storyId']):
soup = BeautifulSoup(ek.get_news_story(story))
links=[]
for a in soup.find_all('a', href=True):
links.append(a['href'])
df['Links'][idx] = links
df

I've added the output as a picture.

Any help would be great. Thanks.

Jirapongse

@s.tank

Thank you for reaching out to us.

The code could be like this:

df = ld.news.get_headlines(
    'Topic:GB AND (SOURCE:CMPNY OR SOURCE:MCE OR SOURCE:GEN)',
    start='2025-07-23',
    end='2025-07-29')

df['Links']=""
df['Story']=""
for idx, story in enumerate(df['storyId']):
    story_text = ld.news.get_story(story, format=ld.news.Format.TEXT)
    soup = BeautifulSoup(ld.news.get_story(story,  format=ld.news.Format.HTML))
    links=[]
    for a in soup.find_all('a', href=True):
        links.append(a['href'])    
    df['Links'][idx] = links
    df['Story'][idx] = story_text

df

I use the LSEG Data Library for Python.

The news example is on GitHub.

wasin.w V3

Hello @s.tank

This https://developers.lseg.com/en/article-catalog/article/Upgrade-from-using-Eikon-Data-API-to-the-Data-library might help you upgrade from the Eikon Data API to the Data Library.

s.tank

https://community.developers.refinitiv.com/discussion/comment/127287#Comment_127287

Hi Jirapongse

I had tried this before using ld.news.get_story but it keeps running into issues. I have attached the error message I receive. Below are the list of my imports I have before any code.

import re
import pandas as pd
from bs4 import BeautifulSoup
import requests
import bs4
import html5lib
import eikon as ek
import numpy as np
import configparser as cp
import lseg.data as ld
import shutil
import openpyxl
from openpyxl import load_workbook
from openpyxl.utils.dataframe import dataframe_to_rows
import pyodbc
import sqlite3
import html2text
from IPython.display import HTML
from IPython import get_ipython
cfg = cp.ConfigParser()
cfg.read('eikon.cfg')
ek.set_app_key('***')

ld.open_session()

I've added *** instead of my appkey here.

Am I missing an import? Thanks

s.tank

Hi Wasin.

Thanks for the link, I had gone through these links beforehand but was running into the error I mentioned in my response to Jirapongse. Thanks for the help.

wasin.w V3

Hello @s.tank

You cannot set the app-key to Data library via ek.set_app_key() method. They are different library and not share session among each other.

To use a Data Library with the Desktop session, there are two ways.

Programmatic Way

You can set the app-key in the ld.open_session() method directly.

import lseg.data as ld

APP_KEY = 'Your App-Key'
ld.open_session(app_key = APP_KEY)

You can see more example on GitHub: Session example.

Configuration file

Create a file name lseg-data.config.json in the same location as your application (Python code or Notebook file).

{

    "logs": {

        "level": "debug",

        "transports": {

            "console": {

                "enabled": false

            },

            "file": {

                "enabled": false,

                "name": "lseg-data-lib.log"

            }

        }

    },

    "sessions": {

        "default": "desktop.workspace",

        "desktop": {

            "workspace": {

                "app-key": "Your App-Key"

            }

        }

    }

}

Then, just use the ld.open_session() method to initialize the session

import lseg.data as ld

ld.open_session()

s.tank

https://community.developers.refinitiv.com/discussion/comment/127296#Comment_127296

Hi Wasim, thats for the app key information It is working now. I've edited the code so that it also searched the story text for a key word (drop). Below is my code.

df = ld.content.news.headlines.Definition(
query="Topic:GB AND (SOURCE:CMPNY OR SOURCE:MCE OR SOURCE:GEN) AND drop", extended_params={"searchIn": "FullStory"}
).get_data()
df.data.df

I have attached the output. Do you know why this isn't working and showing more results?

Screenshot 2025-08-08 142942.png

Jirapongse

I got four news headlines.

The depth of news history is 15 month.