Hi @Edwin.Kwan ,

Since I don't know what is the API call and library you are using, below I will provide a general workflow for ingesting larger amount of data iteratively. So as shown below you can wrap your code under try except statement and introduce a while loop to retry a couple of times which will help you getting the data you are after more smoothly.

max_steps = 3
step = 0
while step < max_steps:
    try:
        # your API call goes here
        break
 
    except rd.errors.RDError as e: 
        print("RDError code :", e.code) 
        print("RDError message:", e.message)
        step +=1 
        continue

Hope this helps.

Best regards,

Haykaz