Adding story text to output using Python API

Options
s.tank
s.tank Newcomer

Hi

I currently have the following code that is fetching specific new articles using Python API. However I am not sure how to add the story text to the output. I currently have the Headline, StoryId, and Link to the article.

Code:

df = ek.get_news_headlines(
'Topic:GB AND (SOURCE:CMPNY OR SOURCE:MCE OR SOURCE:GEN)',
date_from='2025-07-23',
date_to='2025-07-30')
df

df['Links']=""
for idx, story in enumerate(df['storyId']):
soup = BeautifulSoup(ek.get_news_story(story))
links=[]
for a in soup.find_all('a', href=True):
links.append(a['href'])
df['Links'][idx] = links
df

I've added the output as a picture.

Any help would be great. Thanks.

Screenshot 2025-08-06 091317.png

Answers

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @s.tank

    Thank you for reaching out to us.

    The code could be like this:

    df = ld.news.get_headlines(
        'Topic:GB AND (SOURCE:CMPNY OR SOURCE:MCE OR SOURCE:GEN)',
        start='2025-07-23',
        end='2025-07-29')
    
    df['Links']=""
    df['Story']=""
    for idx, story in enumerate(df['storyId']):
        story_text = ld.news.get_story(story, format=ld.news.Format.TEXT)
        soup = BeautifulSoup(ld.news.get_story(story,  format=ld.news.Format.HTML))
        links=[]
        for a in soup.find_all('a', href=True):
            links.append(a['href'])    
        df['Links'][idx] = links
        df['Story'][idx] = story_text
    
    df
    

    I use the LSEG Data Library for Python.

    The news example is on GitHub.

  • Hello @s.tank

    This https://developers.lseg.com/en/article-catalog/article/Upgrade-from-using-Eikon-Data-API-to-the-Data-library might help you upgrade from the Eikon Data API to the Data Library.

  • s.tank
    s.tank Newcomer

    Hi Jirapongse

    I had tried this before using ld.news.get_story but it keeps running into issues. I have attached the error message I receive. Below are the list of my imports I have before any code.

    import re
    import pandas as pd
    from bs4 import BeautifulSoup
    import requests
    import bs4
    import html5lib
    import eikon as ek
    import numpy as np
    import configparser as cp
    import lseg.data as ld
    import shutil
    import openpyxl
    from openpyxl import load_workbook
    from openpyxl.utils.dataframe import dataframe_to_rows
    import pyodbc
    import sqlite3
    import html2text
    from IPython.display import HTML
    from IPython import get_ipython
    cfg = cp.ConfigParser()
    cfg.read('eikon.cfg')
    ek.set_app_key('***')

    ld.open_session()

    I've added *** instead of my appkey here.

    Am I missing an import? Thanks

  • s.tank
    s.tank Newcomer

    Hi Wasin.

    Thanks for the link, I had gone through these links beforehand but was running into the error I mentioned in my response to Jirapongse. Thanks for the help.

  • Hello @s.tank

    You cannot set the app-key to Data library via ek.set_app_key() method. They are different library and not share session among each other.

    To use a Data Library with the Desktop session, there are two ways.

    Programmatic Way

    You can set the app-key in the ld.open_session() method directly.

    import lseg.data as ld
    
    APP_KEY = 'Your App-Key'
    ld.open_session(app_key = APP_KEY)
    

    You can see more example on GitHub: Session example.

    Configuration file

    Create a file name lseg-data.config.json in the same location as your application (Python code or Notebook file).

    {
    "logs": {
    "level": "debug",
    "transports": {
    "console": {
    "enabled": false
    },
    "file": {
    "enabled": false,
    "name": "lseg-data-lib.log"
    }
    }
    },
    "sessions": {
    "default": "desktop.workspace",
    "desktop": {
    "workspace": {
    "app-key": "Your App-Key"
    }
    }
    }
    }

    Then, just use the ld.open_session() method to initialize the session

    import lseg.data as ld
    
    ld.open_session()
    
  • s.tank
    s.tank Newcomer

    Hi Wasim, thats for the app key information It is working now. I've edited the code so that it also searched the story text for a key word (drop). Below is my code.

    df = ld.content.news.headlines.Definition(
    query="Topic:GB AND (SOURCE:CMPNY OR SOURCE:MCE OR SOURCE:GEN) AND drop", extended_params={"searchIn": "FullStory"}
    ).get_data()
    df.data.df

    I have attached the output. Do you know why this isn't working and showing more results?