Using for loop to get news headlines of multiple companies

I am trying to use for loop to get ESG news headlines for a list of 405 companies.
My code has proven succesfull when retrieving RICs based on ISINs, however, when I apply the same approach to retrieve the news headlines, I get the error "query must be string ". When I try to use "str()" to convert the values into string, and run the for loop, I get the error below:
EikonError: Error code 500 | Backend error. Failed to deserialize backend response. Expected valid JSON. Error: invalid character 'i' looking for beginning of value
See example below of the instance where my for loop code works with the "ek.get_symbology" function, and how it fails with the "ek.get_news_headlines" function.
Succesful example with ek.get_symbology:
## Ensuring that all ISIN ID's in the Brands data frame are strings (or otherwise converting them)
Brands["ID (ISIN)"] = Brands["ID (ISIN)"].astype(str)
## Creating a list of all the ISINs in order to make the RIC call on the API
ISIN_List = Brands["ID (ISIN)"].tolist()
## Dividing the list of 450 compaies into chunks for the API to be able to process it
chunklist = chunk(ISIN_List, 100)
##Calling RICs from API based on ISIN's (with a for loop to batch the request in chunks)
content_df = []
for subs in chunklist:
RICs = ek.get_symbology(subs, from_symbol_type='ISIN', to_symbol_type= 'RIC')
content_df.append(RICs)
content_df = pd.concat(content_df)
Failed attempt with the "ek.get_news_headlines" function.
##Adding the "R:" for insturment clarification + ESG as a topic for the headline retriever
Brands_M["H_Rqst"] = "R:"+ Brands_M["RIC"].map(str) + " AND Topic:ESG"
## Ensuring that all Headline Request in the Brands data frame are strings (or otherwise converting them)
Brands_M["H_Rqst"] = Brands_M["H_Rqst"].astype(str)
##Creating list of RICs codes with ESG topic to call on the API
H_Q_List = Brands_M["H_Rqst"].tolist()
##Dividing the list into chunks for the API to process it
H_Q_chunklist = chunk(H_Q_List, 50)
##Retreiving headlines data based on RICs + ESG as a Topic
Headlines_df = []
for H_Q_subs in H_Q_chunklist:
News = ek.get_news_headlines(query=H_Q_subs,count=10,date_from='2020-09-01T09:00:00',date_to='2020-09-25T18:00:00')
Headlines_df.append(News)
Headlines_df = pd.concat(Headlines_df)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-13-dbae4b732937> in <module>
3 Headlines_df = []
4 for H_Q_subs in H_Q_chunklist:
----> 5 News = ek.get_news_headlines(query=H_Q_subs,count=10,date_from='2020-09-01T09:00:00',date_to='2020-09-25T18:00:00')
6 Headlines_df.append(News)
7
~/opt/anaconda3/lib/python3.7/site-packages/eikon/news_request.py in get_news_headlines(query, count, date_from, date_to, raw_output, debug)
97 error_msg = 'query must be a string'
98 logger.error(error_msg)
---> 99 raise ValueError(error_msg)
100
101 if type(count) is not int:
ValueError: query must be a string
Hope you can help, I suspect it has something to do with the addition of the topic.
Best Answer
-
@Erik77 does this work for you?
df1 = pd.DataFrame()
ricList = ["AAPL.O","IBM.N","TSLA.O"]
for ric in ricList:
q="R:" + ric + " AND Topic:ESG"
print(q)
df = ek.get_news_headlines(q)
df1 = df1.append(df)0
Answers
-
@Erik77 I think your issue may have something to do with proper escaping of the query string - please see my answer on this thread. Could you please post a completed example of the query string: eg print("R:"+ Brands_M["RIC"].map(str) + " AND Topic:ESG")
this works for me:
q="R:" + "IBM" + " AND Topic:ESG"
df = ek.get_news_headlines(q)
dfI hope this can help.
0 -
Thanks for your reply, I am looking into the escaping. In the meantime, see below the print you requested:
0 -
@Erik77 So this query looks fine it just needs to be escaped itself eg "R:MCD AND Topic:ESG"
Can you try:
Brands_M["H_Rqst"] = Brands_M["H_Rqst"].astype(str).values
0 -
This is already the case, when I pass the column as a list, and then into chunks. Yet I still seem to get the error:
Or am I missing something?
0 -
@Erik77 - they need to be escaped with " not ' i believe this is the issue.
0 -
@Erik77 actually I just tried with single quotes and it works for me:
Can you try removing all the other parts of the query eg count and start/end date and try again? Does the below work for you:
q='R:MCD AND Topic:ESG'
df = ek.get_news_headlines(q)
df0 -
Yes it does, thank you!
0
Categories
- All Categories
- 3 Polls
- 6 AHS
- 36 Alpha
- 166 App Studio
- 6 Block Chain
- 4 Bot Platform
- 18 Connected Risk APIs
- 47 Data Fusion
- 34 Data Model Discovery
- 685 Datastream
- 1.4K DSS
- 615 Eikon COM
- 5.2K Eikon Data APIs
- 10 Electronic Trading
- Generic FIX
- 7 Local Bank Node API
- 3 Trading API
- 2.9K Elektron
- 1.4K EMA
- 252 ETA
- 556 WebSocket API
- 38 FX Venues
- 14 FX Market Data
- 1 FX Post Trade
- 1 FX Trading - Matching
- 12 FX Trading – RFQ Maker
- 5 Intelligent Tagging
- 2 Legal One
- 23 Messenger Bot
- 3 Messenger Side by Side
- 9 ONESOURCE
- 7 Indirect Tax
- 60 Open Calais
- 275 Open PermID
- 44 Entity Search
- 2 Org ID
- 1 PAM
- PAM - Logging
- 6 Product Insight
- Project Tracking
- ProView
- ProView Internal
- 22 RDMS
- 1.9K Refinitiv Data Platform
- 652 Refinitiv Data Platform Libraries
- 4 LSEG Due Diligence
- LSEG Due Diligence Portal API
- 4 Refinitiv Due Dilligence Centre
- Rose's Space
- 1.2K Screening
- 18 Qual-ID API
- 13 Screening Deployed
- 23 Screening Online
- 12 World-Check Customer Risk Screener
- 1K World-Check One
- 46 World-Check One Zero Footprint
- 45 Side by Side Integration API
- 2 Test Space
- 3 Thomson One Smart
- 10 TR Knowledge Graph
- 151 Transactions
- 143 REDI API
- 1.8K TREP APIs
- 4 CAT
- 27 DACS Station
- 121 Open DACS
- 1.1K RFA
- 104 UPA
- 193 TREP Infrastructure
- 228 TRKD
- 917 TRTH
- 5 Velocity Analytics
- 9 Wealth Management Web Services
- 90 Workspace SDK
- 11 Element Framework
- 5 Grid
- 18 World-Check Data File
- 1 Yield Book Analytics
- 46 中文论坛