Uncaught UnicodeEncodeError on Python 2 in debug statement
In Eikon Python 1.0.1, at eikon.json_requests line 114, there is a call to logger.debug('HTTP Response:{}'.format(response.text)) that is not wrapped in a try: except UnicodeEncodeError call (as is the case with most other calls, see e.g. lines 90-94). If the HTTP request returns a value with a non-ASCII Unicode value this then causes an uncaught exception to raise in Python 2. In Python 3.6+ this is not an issue as the non-ASCII unicode value is handled by the Python 3 str.format method.
Hello @davidk
Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?
If yes, please click the 'Accept' text next to the reply. This will guide all community members who have a similar question. Otherwise please post again offering further insight into your question.
Thanks,
AHS
This bug has not been fixed, even though I explained above how to patch the problem (by simply wrapping json_requests.py line 114 in a try: except UnicodeEncodeError: block).This should have been an easy patch to create a 1.0.2. In our firm we have distributed the fix but others should receive it as well. If Eikon wishes not to support bugfixes on python 2.7 given python 2.7's end-of-life on Jan 1, 2020, that is fine, but they should say so if so.
Hi @davidk
The defect was reported to development team and JIRA was created since you reported it.
But we have not been informed on any release cycle yet.
As I noted in the original question, this was not an issue in Python 3.6+. It was only an issue in Python 2 where Unicode had to be explicitly decoded. Since Python 2 went end of life exactly a year ago there is no longer a need to fix this.
Hi @davidk
I think I can reproduce the issue.
I will consult with my senior colleague and will give you an update.
This is not quite a reproduction-- I think you're hitting a different logger line that is wrapped in a try: except: (likely line 92), rather than line 114 (otherwise, you would have received a raised exception). In order to verify this, in a new python window run:
import logging; logging.basicConfig(level=logging.DEBUG)
Before importing the eikon API. You should then see the log messages generated and the line they came from.
In order to hit the error on line 114 you need to have received a ticket from DataGrid. Based on its behavior I believe the ticket is sent if the server warns you that there will be a delay in the response. For a true reproduction therefore you might need to send a particularly large request (say, for a few thousand RICs for several fields).
I suggest you simply review the source code, it's quite clear what the problem is. All the other logger.debug('HTTP Response: {}'.format(response.text)) calls in json_requests.py are wrapped in try: except UnicodeEncodeError: other than the one on line 114.
Basically if you hit one of the other calls, you get the pyeikon message you received above (if you haven't initialized the root logger) or an error message saying that Unicode parsing failed (if you have initialized the root logger). But if you get a ticket from DataGrid first and THEN hit the Unicode error on line 114, you get an exception and the program blows its stack.
Sorry to be cagey about reproduction but reproducing once requires running enough queries to generate a DataGrid ticket and that depends on the server load. We were hitting this error once every 1-2 hours while downloading analyst recommendations for a quant analysis we were running, since many of the analyst's names contain accented Unicode characters. So basically if you really want to reproduce, you can try running this code on a loop for several hours:
import logging
logging.basicConfig(level=logging.DEBUG)
import eikon
rics = eikon.get_data('0#.SPX', 'TR.IndexConstituentRIC')[0].iloc[:, 0].tolist()
fields = ['TR.RecEstValue.value', 'TR.RecEstValue.brokername',
'TR.RecEstValue.brokerid', 'TR.RecEstValue.analystname',
'TR.RecEstValue.analystcode', 'TR.RecEstDate']
while True:
eikon.get_data(rics, fields)
