For a deeper look into our Eikon Data API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials |  Articles

question

Upvotes
Accepted
244 9 11 16

Uncaught UnicodeEncodeError on Python 2 in debug statement

In Eikon Python 1.0.1, at eikon.json_requests line 114, there is a call to logger.debug('HTTP Response:{}'.format(response.text)) that is not wrapped in a try: except UnicodeEncodeError call (as is the case with most other calls, see e.g. lines 90-94). If the HTTP request returns a value with a non-ASCII Unicode value this then causes an uncaught exception to raise in Python 2. In Python 3.6+ this is not an issue as the non-ASCII unicode value is handled by the Python 3 str.format method.

eikoneikon-data-apipythonworkspacerefinitiv-dataplatform-eikonworkspace-data-apiexceptionunicode
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hello @davidk

Thank you for your participation in the forum. Is the reply below satisfactory in resolving your query?

If yes, please click the 'Accept' text next to the reply. This will guide all community members who have a similar question. Otherwise please post again offering further insight into your question.

Thanks,

AHS

davidk avatar image davidk pimchaya.wongrukun01

This bug has not been fixed, even though I explained above how to patch the problem (by simply wrapping json_requests.py line 114 in a try: except UnicodeEncodeError: block).This should have been an easy patch to create a 1.0.2. In our firm we have distributed the fix but others should receive it as well. If Eikon wishes not to support bugfixes on python 2.7 given python 2.7's end-of-life on Jan 1, 2020, that is fine, but they should say so if so.

Hi @davidk

The defect was reported to development team and JIRA was created since you reported it.

But we have not been informed on any release cycle yet.

Upvotes
Accepted
244 9 11 16

As I noted in the original question, this was not an issue in Python 3.6+. It was only an issue in Python 2 where Unicode had to be explicitly decoded. Since Python 2 went end of life exactly a year ago there is no longer a need to fix this.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
18.1k 21 12 20

Hi @davidk

I think I can reproduce the issue.

I will consult with my senior colleague and will give you an update.


ahs.png (183.9 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

This is not quite a reproduction-- I think you're hitting a different logger line that is wrapped in a try: except: (likely line 92), rather than line 114 (otherwise, you would have received a raised exception). In order to verify this, in a new python window run:

import logging; logging.basicConfig(level=logging.DEBUG)

Before importing the eikon API. You should then see the log messages generated and the line they came from.

In order to hit the error on line 114 you need to have received a ticket from DataGrid. Based on its behavior I believe the ticket is sent if the server warns you that there will be a delay in the response. For a true reproduction therefore you might need to send a particularly large request (say, for a few thousand RICs for several fields).

Hi @davidk

Can you provide the code which you use to reproduce the issue ?

Thanks a lot.

I suggest you simply review the source code, it's quite clear what the problem is. All the other logger.debug('HTTP Response: {}'.format(response.text)) calls in json_requests.py are wrapped in try: except UnicodeEncodeError: other than the one on line 114.

Basically if you hit one of the other calls, you get the pyeikon message you received above (if you haven't initialized the root logger) or an error message saying that Unicode parsing failed (if you have initialized the root logger). But if you get a ticket from DataGrid first and THEN hit the Unicode error on line 114, you get an exception and the program blows its stack.

Sorry to be cagey about reproduction but reproducing once requires running enough queries to generate a DataGrid ticket and that depends on the server load. We were hitting this error once every 1-2 hours while downloading analyst recommendations for a quant analysis we were running, since many of the analyst's names contain accented Unicode characters. So basically if you really want to reproduce, you can try running this code on a loop for several hours:

import logging

logging.basicConfig(level=logging.DEBUG)

import eikon

rics = eikon.get_data('0#.SPX', 'TR.IndexConstituentRIC')[0].iloc[:, 0].tolist()

fields = ['TR.RecEstValue.value', 'TR.RecEstValue.brokername',
          'TR.RecEstValue.brokerid', 'TR.RecEstValue.analystname',
          'TR.RecEstValue.analystcode', 'TR.RecEstDate']

while True:

   eikon.get_data(rics, fields)
Upvotes
16.7k 31 9 12

Hello @davidk

Do you still encounter the issue with the latest version of Eikon Data API and Python 3.x?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.