question

Upvotes
Accepted
23 0 0 2

Restarting session after 401 error in Python

I'm using a basic python setup, following the session example on GitHub.


  
                
  1. import refinitiv.data as rd
  2. from refinitiv.data.content import fundamental_and_reference
  3.  
  4. rd.open_session()
  5.  
  6. # run routines ->

After a number of routines, some 1000 correct calls with http 200 I sometimes receive http code 401. Token de-activated. The only way to solve for this is to restart the script.

My question: How can I programmatically restart the session.


I would expect something as:

  
                
  1. response = fundamental_and_reference.Definition(universe=['example']).get_data()
  2.  
  3. if response._http_status['http_status_code'] == 401:
  4. # solution like this would be nice:
  5. rd.restart_session()


rd.restart_session obviously does not exist. rd.close_session() and rd.open_session() does not work as well. The code freezes and does not continue.


In the previous answer I got referred to the GitHub, but there is no code explaining there what to do.


What can we do?

python#technology
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
Accepted
6.2k 21 3 6

Hi @r.fernandez,


We advise coding defensively in these instances. I manufactured an error in the code below to exemplify what I mean:


import refinitiv.data as rd
from refinitiv.data.content import fundamental_and_reference

rd.open_session()

request = fundamental_and_reference.Definition(
    universe=['LSEG.L'],
    fields=['ASK'])


try:
    response = request.get_data()
except Exception as e:
    print(e.__dict__)


try:
    response = request.get_data()
except Exception as e:
    if e.__dict__['code'] == -1:
        print("The Error Is -1")


1716804033978.png


As you can see, you can use a try and if loop to come to a similar functionality to what you asked. Please let me know if this makes sense.


1716804033978.png (65.2 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Hi @jonathan.legrand

This does not answer my question. Obviously in the production code we have a try/except around it.


My very very specific question: how do I restart the session after receiving a 401 error?


Can you please think about an answer and email this to me? Thanks

Hi @r.fernandez, If you would like to keep your code lines' framework, you could try something like this:


if response._http_status['http_status_code'] == 401:

    rd.close_session()
    rd.open_session()
    response = fundamental_and_reference.Definition(universe=['example']).get_data()


If you often encounter this issue, I would advise making a function to capture all calls:


import time

def get_fundamental_and_reference_data(
        universe,
        fields,
        max_attempts=4,
        parameters=None,
        row_headers=None,
        use_field_names_in_headers=None,
        extended_params=None
        ):
    
    for i in range(max_attempts):
        response = rd.content.fundamental_and_reference.Definition(
            universe=universe,
            fields=fields,
            parameters=parameters,
            row_headers=row_headers,
            use_field_names_in_headers=use_field_names_in_headers,
            extended_params=extended_params).get_data()
        if response._http_status['http_status_code'] == 401:
            time.sleep(0.5)
            rd.close_session()
            time.sleep(2.5)
            rd.open_session()
            time.sleep(1)
        elif response._http_status['http_status_code'] != 401:
            return response

Hi @jonathan.legrand


As I wrote in my question: d.close_session() and rd.open_session() does not work as well. The code freezes and does not continue.

I will show you the code I have (similar to your code I've created a function to capture all calls). The error handling specifically for 401 can be written out with time.sleep in between:

schermafbeelding-2024-05-29-om-133220.png


This results in the following routine. As you see in the command line, the code runs perfectly, and after about 10 minutes (approximately), the returned error is 401. As you can see, I've setup the routine, and as you see the code freezes on "rd.open_session()"

schermafbeelding-2024-05-29-om-133114.png

I've waited for over 5 minutes, no result. The code freezes.


Can you come up with a method which does not result in this behavior?


Show more comments
Upvotes
23 0 0 2

@jonathan.legrand

Did you receive the message yesterday on the error code 401 and the script? Have you been able to replicate the error code after running it for some ten minutes?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

Upvotes
23 0 0 2

@jonathan.legrand

I've answered your thread last week, but have not received a reaction. Could you please take a look at it? My note:

The error I receive is: "Error code 401 | token expired"

I've replicated the full code below, based on your example. I've ran the code in 3 command windows in parallel to replicate the production environment. After 1975 calls, 4 minute 20 seconds, I receive this Exception and the code freezes.

Note: Your code you check 401 in the http response, but in practice 401 raises an error and thus must be caught using try/catch.

See code:

import refinitiv.data as rd
import time
from datetime import datetime 
import random

rd.open_session()
start = datetime.now()

def get_fundamental_and_reference_data(
        universe,
        fields,
        max_attempts=4,
        parameters=None,
        row_headers=None,
        use_field_names_in_headers=None,
        extended_params=None,
        ncalls = 0
        ):

    for i in range(max_attempts):
        
        try:
            response = rd.content.fundamental_and_reference.Definition(
            universe=universe,
            fields=fields,
            parameters=parameters,
            row_headers=row_headers,
            use_field_names_in_headers=use_field_names_in_headers,
            extended_params=extended_params).get_data()
            return response._http_status['http_status_code']
        except Exception as e:
            error = str(e).split('Requested')[0]
            
            # check if 401 error, if so restart and fully exit script
            if 'code 401' in error or 'token expired ' in error:
                print(f'\n\n-> Error 401: {error}')
                print(e)
                print('Script started at: ', start)
                print('Current time: ', datetime.now())
                print('Number of calls done: ', ncalls)
                
                time.sleep(1)
                print(f'{datetime.now()} -- closing session')
                rd.close_session()
                print(f'{datetime.now()} -- session closed')
                time.sleep(2.5)
                print(f'{datetime.now()} -- opening new session')
                rd.open_session()
                print(f'{datetime.now()} -- session opened')
                time.sleep(1)
                print(f'{datetime.now()} -- continuing')
            
            return e

ncalls = 0
while True:
    ncalls += 1
    http=get_fundamental_and_reference_data(universe="VOD.L", fields="TR.ClosePrice", ncalls=ncalls)
    print(f'    - call {ncalls} -- http: {http}', end='\r')
    
    #  to ensure no 429 data limiting calls
    time.sleep(random.uniform(0, 0.5))


FYI, reran the code on Monday 3 June. Performed the code in 2 command windows, and first window received 401 error after just 64 calls. See screenshot:

schermafbeelding-2024-06-03-om-125942.jpg


Running on Mac OSX, latest version of osx, python 3.9 and pip version.


icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.

The discrepancy between your experience and mine is making me think that this may be a permission issue, whereby you may be using credentials aimed solely at Desktop Sessions and not Platform Sessions.
The code you kindly shared with us includes the line `rd.open_session()`; may I know what session you have as default in your configuration file? Is it Desktop Sessions? Or maybe you do not use a configuration file? Because in this case, what may be happening is that the proxy on Workspace may be shutting down after a long while of inactivity on the Desktop Application or after a session limit, leading to the issues you are experiencing. In this case, a solution would be to shut down and restart the Workspace Desktop Application every time your code runs into this issue. If your LSEG License allows for Platform Sessions, I would advise using this in your instance; you may be requesting a lot of data, which reflects the use of Platform Sessions for which these were made.


Write an Answer

Hint: Notify or tag a user in this post by typing @username.

Up to 2 attachments (including images) can be used with a maximum of 512.0 KiB each and 1.0 MiB total.