Sentiment does not correspond to currency pair in filter.

azzopardic

I'm currently working on a market sentiment report and utilizing Python's get_news_headlines function to collect data. Because the function is run for each currency pair, if more than one currency pair is referenced in the same news, there may be duplicates. The dashboard shows the news' sentiment and how it relates to the currency pair in question. The problem is that there are times when a news piece discusses only one currency pair, and the sentiment of that news article is directed toward that currency pair, but it also includes mentions to other currency pairs. Hence, when the filters for the other currency pairs are applied, this news story is also caught. As a result, the sentiment gathered in this news story does not correspond to the other currency pairs indicated, causing the statistics gathered to be erroneous. Is there any way to resolve this problem, please?

Examples of filters used currently: EURGBP=, EUR=, EURJPY=

Example of this issue:

Thanks in advance.

jason.ramchandani01

@azzopardic So this is essentially an NLP (Natural Language Processing) and classification problem set - not really an API issue. So if I am right you are looking to separate articles that only talk about one currency pair and ignore articles that mention more than one currency pair. In this case you would need to download the news story and then scan for mentions of any other pairs (ie you should have a list of all pairs) and then only pass the news item to the sentiment engine if only the pair you are referring to is mentioned or you could say then weight the sentiment of the article according to the number of mentions of a currency pair - please see the following example:

from IPython.display import HTML
from bs4 import BeautifulSoup

df = ek.get_news_headlines('GBP=',count=100)
df

this returns us a list of headlines and storyIds. So we then download a news story like this:

We could then count the number of mentions of particular phrases - in our case from currency pairs or rics even:

fx_list = ['GBP','JPY','USD','EUR','EUR/GBP','GBP/EUR']


for fx in fx_list:
    soup = BeautifulSoup(story,"html")
    wordcount = sum(fx == word for word in soup.get_text().split())
    print(fx, wordcount)

So once you have this data you could then decide whether the article was more about one pair or more about other pairs. I don't think this would be the holy grail but its a simple way for you to perhaps experiment a bit to see how things are working in real life. I hope this simple sample can help.