Lseg-data.get_data slow after 3pm UTC

Huong

Observing Lseg-data API getting slower after 3pm UTC.

There are occasional exceptions thrown when calling lseg.data.get_data() function. Exceptions listed below

1. lseg.data._errors.LDError: [WinError 10061] No connection could be made because the target machine actively refused it
2. lseg.data._errors.LDError: Asynchronous Query library internal error.
3. lseg.data._errors.LDError: timed out

We have also seen logs where the response took around an hour, is this expected? shouldn't the request have timed out in such case? Is there a way for lseg.data.get_data() to timeout if it is taking more than a configured amount of time to respond?

Is there any known time period of the day when the API is expected to be slow? We don't encounter above issues if same API with same param are called during the first half of the day or before 3pm UTC.

Jirapongse

@Huong

Thank you for reaching out to us.

Regarding the library's response time, this will depend on the amount of data, the complexity of your request, and the server load. The amount of time is usually a result of processing time in the backend as opposed to the library itself.

To verify what the problem is, you may need to enable the debug log in the library by using the following code before calling the open_session method.

config = ld.get_config()

config.set_param("logs.transports.file.enabled", True)

config.set_param("logs.transports.file.name", "lseg-data-lib.log")

config.set_param("logs.level", "debug")

With these configurations, the library will create a lseg-data-lib.log file. We could verify the behavior of the library via this log file.

Huong

https://community.developers.refinitiv.com/discussion/comment/123649#Comment_123649

For responses taking too much time, is there a way to timeout/cancel such requests? We have observed that when the response time is unusually long (say 15 minutes to 1 hour), the response contains incomplete data (some of the requested fields are empty). The same request when send at a different time completes in 2 min and contains complete data.

If the requests would timeout if taking more than usual time, we could do a retry. Is this possible within Lseg.data library or we need to handle at our/caller side (discard the request after a configured time lapse and start a new one)

Huong

https://community.developers.refinitiv.com/discussion/comment/123649#Comment_123649

We observed the lseg.data.get_data() responding back with "Asynchronous Query library internal error" for every request from the night of 17/01/2025 onward. The same requests were working fine earlier (before the night of 17/01/2025) and when checked today, they are working fine.
Our requests are straight forward, we request for either prices or descriptive data for a specific set of bonds (around 5000) or equities (around 6000). We request in batches of 500 securities (ISIN) at a time.
Below are some library debug logs

20250118-0459-145868-lseg-data-lib.log

20250118-0500-144808-lseg-data-lib.log

20250120-0353-146420-lseg-data-lib.log

All requests done on the night of 17/01/2025 and morning/noon of 18/01/2025 failed with the error
"Asynchronous Query library internal error".
20250118-0459-145868-lseg-data-lib.log → Price field request for 500 ISINs with failed response.
20250118-0500-144808-lseg-data-lib.log → Price field request for 4 ISINs with failed response.
20250120-0353-146420-lseg-data-lib.log → Price field request for 500 ISINs with successful response.

Is this a server load issue? If so can it happen anytime or we have know time period when this is expected?

Can anything be done from our side to avoid/overcome this? 17/01/2025 (night) to 18/01/2025 is a very long period for the API to be unavailable.

Jirapongse

@Huong

Thank you for the log files.

I found that the errors are in the response sent by the server or endpoint.

It is not an issue in the LSEG Data library. Please contac the helpdesk team directly via MyAccount to verify this issue on the server side.