Why do I keep getting timeout errors when pulling the full list of US stocks? Are there any solution

rice

cond = 'IN(TR.ExchangeCountryCode,US)'
each_req = "SCREEN(U(IN(Equity(active,public)))/UNV:Public/, " + cond + ", NOT_IN(TR.InstrumentTypeCode,BONDFUT,BONDSPREAD,CEF,ETF,ETFA,ETFB,ETFC,ETFE,ETFM,ETFO,ETFX,ETMF,FUN,GROWUNT,HDG,INS,OPF,OPTRTS,PAIDSUBRTS,PREFERRED,PRF,RTS,SUBSRTS),CURN=USD)"
fields = ["TR.OrganizationID", "TR.CompanyName", "TR.ExchangeName", "TR.IsPrimaryInstrument", "TR.ISIN","TR.InstrumentType", "TR.ExchangeCountry"]
df_stock = ld.get_data(each_req, fields)

jason.ramchandani01

@rice So this query works for me (I removed the /UNV:Public/) as you have already specified it:

cond = 'IN(TR.ExchangeCountryCode,US)'

each_req = "SCREEN(U(IN(Equity(active,public)))," + cond + ", NOT_IN(TR.InstrumentTypeCode,BONDFUT,BONDSPREAD,CEF,ETF,ETFA,ETFB,ETFC,ETFE,ETFM,ETFO,ETFX,ETMF,FUN,GROWUNT,HDG,INS,OPF,OPTRTS,PAIDSUBRTS,PREFERRED,PRF,RTS,SUBSRTS),CURN=USD)"

fields = ["TR.OrganizationID", "TR.CompanyName", "TR.ExchangeName", "TR.IsPrimaryInstrument", "TR.ISIN","TR.InstrumentType", "TR.ExchangeCountry"]


df_stock = rd.get_data(each_req, fields)

Does this work for you? It is also a large universe being returned - the indicated limit per API call is 10K datapoints per API call - this is over 100k (ie 10x the recommended amount). It is not surprising that you are getting timed out - particularly if you are running it during periods of high server load. An idea to return smaller chunks of the universe would be to split by market cap band, or by company name beginning with a letter of alphabet. The benefit of this with large universes is less timeouts. I hope this can help.

Kindest regards,

rice

https://community.developers.refinitiv.com/discussion/comment/127357#Comment_127357

Thank you for the suggestion. I’d like to try splitting the queries by market cap bands and company name initials. Could you provide a code example for this?

Jirapongse

@rice

You can contact the helpdesk team via LSEG Support and ask for the screener formula which can be used to get the required item.

Then, you can apply the formula to the Python code. For more information, please refer to this Find Your Right Companies with SCREENER | the Data Library (Python) article.