Different results from get_data request depending on session type

robin.ameloot
robin.ameloot Newcomer
in Refinitiv Data Platform

Hi all,


I was previously helped by being shown the difference of a desktop session vs platform session and wanted to convert my rd.get_data() requests to exploit the benefits of a platform session over a desktop session.

In doing so I've encountered two anomalies I wanted to see if I could get some clarity on.

1) Parameters

Using the same list of fields and associated parameters resulted in multiple errors. As I'm working off old code form a colleague I'm not sure what all the parameters do, but it seems weird that parameters passed using a desktop session cause errors when using a platform session.


I found that 'CH', 'RH', 'NULL' and 'Transpose' throws a 'Error code -1) error.

2)

Omitting these erroneous parameters allowed me to extract the data but I noticed another difference. I'll illustrate with the two sections of code shown below:


1707836654480.png

1707836671573.png

the sole difference (to my understanding) here being that the top snippet is using a default desktop session and the bottom a custom platform session.

The results of these snippets differ in that the top will return all years even if there is no data available (desired output), whereas the botton snippet only returns data for the years where data is available (undesired outcome).

My question is whether these problems are caused by a fault from my side? If not, what is causing this or in what way can I best remedy this?


Thanks in advance!

Tagged:

Best Answer

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    Answer ✓

    @robin.ameloot

    Thank you for reaching out to us.

    Yes, you are correct. The results are different. I used the following code.

    rd.get_data('LVMH.PA',['TR.CashInterestPaid.fperiod','TR.CashInterestPaid'],
                {'SDate':'FY0','EDate':'FY-32','Period':'FY0','Frq':'FY','Scale':6})

    This is the output form the desktop session.

    1707883236842.png

    This is the output from the RDP session.

    1707883308576.png

    However, this developer forum is for questions on and general discussions of Refinitiv APIs.

    To verify the content, please contact the helpdesk team directly via MyRefinitiv.

    I have tried to submit this issue on your behalf but the system didn't create a ticket number. Therefore, please try to submit this issue via MyRefinitiv.

Answers

  • robin.ameloot
    robin.ameloot Newcomer

    For those curious as to what the results look like I've attached the results for a default desktop session below:

    1707840640579.png

    And the results for a custom platform session below:

    1707840607220.png


    the problem caused here is that when extracting multiple variables in a single get_data() call the data and years don't necessarily matchup meaning the data in the dataframe is in essence worthless.

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/114153#Comment_114153

    @robin.ameloot

    Now, I can sumit this query on your behalf. The case number is 13307519.

    The support team will contact you regarding this issue.

  • robin.ameloot
    robin.ameloot Newcomer
    https://community.developers.refinitiv.com/discussion/comment/114155#Comment_114155
    Thank you very much!
  • FFabianB
    FFabianB Newcomer

    @Jirapongse

    I am replying to this thread as customersupport@refinitiv.com is unable to help me on this. I am experiencing a similar issue:

    I run the same code as always, however, this time I tried using the platform session I only received back one field (‘TRDPRC_1’) instead of all requested ones when I do it via desktop session:

    Session was opened as follows:

    “

    session_rda = rd.open_session(config_name="C://Deployment//Daily_Download_Refinitiv//refinitiv-data.config.json", name='platform.rdp')

    ”

    Usually, I try with the below but that has proven unreliable as the session is sometimes not available/able to be opened:

    “

    session_desktop = rd.open_session(config_name="C://Deployment//Daily_Download_Refinitiv//refinitiv-data.config.json", name="desktop.workspace")

    ”

    And then the actual request:

    “

    stock2 = rd.get_data(

        universe=[i for i in updated_rics if i is not None],

        fields= [ 'TR.CLOSEPRICE.date', 'TR.BIDPRICE.date','TR.CLOSEPRICE','TRDPRC_1','TR.SETTLEMENTPRICE','TR.BIDPRICE','TR.ASKPRICE', 'TR.ACCUMULATEDVOLUME', 'TR.VOLUME', 'TR.F.ComShrOutsTot','TR.COMPANYMARKETCAP', 'TR.BetaWkly2y','TR.BetaDaily180D','TR.OPENINTEREST','TR.DELTA','TR.IMPLIEDVOLATILITY'],

        parameters= {'SDate': positionsdate.strftime('%Y-%m-%d'), 'EDate': valuation_date.strftime('%Y-%m-%d'), 'FRQ': 'D'})

    ”

    Why am I only getting back “TRDPRC_1” when using the platform session but all the fields when using the same code via desktop session?

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/122930#Comment_122930

    As far as I know, it uses the different endpoints to retrieve the data. It may also relate to account's permissions or licenses.

    To verify what the problem, you need to enable logging in the RD library by running the following code before opening a session.

    config = rd.get_config()
config.set_param("logs.transports.file.enabled", True)
config.set_param("logs.transports.file.name", "refinitiv-data-lib.log")
config.set_param("logs.level", "debug")
rd.open_session(…)
  • FFabianB
    FFabianB Newcomer
    https://community.developers.refinitiv.com/discussion/comment/122939#Comment_122939
    20241120-0947-17868-refinitiv-data-lib.log
    20241120-0951-24624-refinitiv-data-lib.log

    @Jirapongse Thanks for your reply. I am attaching the logs with the parameters set as suggested. File "0947-17868" is with platform session and the other with desktop session.

    Can you please confirm, what type of permission needs to be checked, so I can reach out to support?

  • Jirapongse
    Jirapongse ✭✭✭✭✭
    https://community.developers.refinitiv.com/discussion/comment/122964#Comment_122964

    @FFabianB

    You requested a lot of items and I found the following error. 

    2024-11-20T09:48:03.886006+01:00] - [sessions.platform.rdp.0] - [ERROR] - [3776 - MainThread] - [http_service] - [request] - An error occurred while requesting URL('https://api.refinitiv.com/data/datagrid/beta1/').
    
ReadTimeout('The read operation timed out')

    It should relate to the HTTP Client Timeout. You may try to increase the timeout via the configuration.


    
config.set_param("http.request-timeout", 6000)
    
rd.open_session(…)

    Otherwise, you can reduce the number of requested instruments.

  • FFabianB
    FFabianB Newcomer
    https://community.developers.refinitiv.com/discussion/comment/122965#Comment_122965

    @Jirapongse Thanks, it seems to be working fine now.

    Is there a writeup available what parameters are available and what the defaults are when using which session?

  • FFabianB
    FFabianB Newcomer
    https://community.developers.refinitiv.com/discussion/comment/122965#Comment_122965

    Hi @Jirapongse, when I ran the script this night, I had a similar issue. Again only one field was downloaded "TRDPRC_1". When I reran the script just now it worked fine again. Can you please advise? Was it again due to the timeout? I cannot see such an error.

    20241121-0737-5184-refinitiv-data-lib.log
    20241121-0300-8772-refinitiv-data-lib.log
  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @FFabianB

    I saw the gateway timeout.

    	url = https://api.refinitiv.com/data/datagrid/beta1/
    
	method = POST
    
	headers = {'x-tr-applicationid': 'RD-PYTHON-LIB', 'Content-Type': 'application/json', 'Authorization': 'Bearer eyJhbGciOiJSUzI1NiIsImtpZCI6ImRMdFd2Q0tCSC1NclVyWm9YMXFod2pZQ2t1eDV0V2ZSS2o4ME9vcjdUY28iLCJ0eXAiOiJhdCtqd3QifQ.eyJhdWQiOiIxOWE1NWNkMmFmNzk0YzMyOTVmMzliOWQ4MmE0MmJhMTVhODQ5YWNmIiwiZGF0YSI6IntcImNpcGhlcnRleHRcIjpcIkxHeWhsNE94eXZyLUxxQ19IUDdqWGtZN3RaNEg3TzV3S3phU2cxYVNzOG1kZWZuaXV3ZW16Z0RYSU1qZzlmMWp0ZkpPODNMVmRNSGMyWHY5R2l1aGI5Tm16WThoQW95TzM1QXB1UG5tbWpNbDEwU1FCVUt3MndBTnE0NUIzTzIyQjJqWjRZbUxGbDRoSzlPRE1tZ05ra0hGZzdEemVmYXBDM3JTR2hFektOVUUyY3NWWUZHYW9hQ2VCV1d5LVdGQ0dUUzdqWkI5R3pMT1BmLTVoRzJNWFUybENTT2VJaWY0eHFtMS1YMjV4UlZBa01zMXlKY0QzUUhab0ZFMWkwRm41elZHdTlmdlhwV3I4Y1hVTTlKZlN6UzMwZktWTkl2d05Cb01lUGpKNmY3ZHVXWXV0TGlxREl0T3ZCZlFoeWp3SWtWTUl4UWRTZGR3VXBxaE80SW1zZjdpbEd3Q2tFS19qcHZKc29VY0NJNkg3dzZMN3M0U1l4S3BXSGoxNlNsakhHSjJ5N3NsNVpKbldzOWdmUlZrTlZ0VWZMUDRsalpjMWRzRXRLT0pKREJicjBhVEdpcTNFQ1ZOVk9PU3BXUnFkUmpNQzlUSzlaV0NBRm90V0NVVHFqWGplMGE1c0ZzblBVYUFHb1kzLXRfUVZKMHlrRmJvdkMtbHFfY2VTZ0hNTi1UUWNrMkNTa3RnSV8yTnlOalFGeGM2RE1FSzNBbldjTnpQLTJDTTQtSnEtcW5pbS1lMXRiTnczbzY2ODd6WUZta0t3UlRqM0tZdFJYbnVRb1oxcnZtZkl0ak9nOHZETHQ3NWx0bll0aWM3bDJmNDZUQnVHRjIwUTBzTUZCR1BhMmVha1BUYzhpRmsweHFZMVdFWXJKdldySlk5NjNKZjNNa2ZVUWdjMUUyOGxILW9rb1M5M3kyRklvRU93MnV1TmVNbENMOWU3d2tuei1oNlRJUV9IeG1iNWtRNF83MU9SWEk5MkdDem5CZFVTU2QzUDgzQ18wUzJjVkxCWTY0NlZaNFBVNDVJWE14cFZxaWctcGI1VlQ5QXUxb3ZBRjY3REVDNjZMZnJwcF83RGxGaklkWWdxNWxvSndfbjZXM2lhUXlJSE00XCIsXCJlbmNyeXB0ZWRfa2V5XCI6XCJBUUlCQUhoa2VfVEpvbngtYkthdzBETWM5RjBWaERYb3dwVFV4dXA3blFTckpfU3Fod0VmakxVYkZSN01MN19kLVg0cFAzQktBQUFBZmpCOEJna3Foa2lHOXcwQkJ3YWdiekJ0QWdFQU1HZ0dDU3FHU0liM0RRRUhBVEFlQmdsZ2hrZ0JaUU1FQVM0d0VRUU10VkxnTHFXRTUydDJDb2U3QWdFUWdEdHFUWjUyWnFrZW14X0ozU2pka1RCSnIyV01iLUhhSHJqN2J0aWluNHJ4ZFUwUDhhX1AtRm1TcGNRRmVOOXQzckxERkRUZ3dqN2JKMG5la0FcIixcImhlYWRlclwiOntcImtpZFwiOlwiYXJuOmF3czprbXM6ZXUtd2VzdC0xOjg5ODA4NDk4NDg3NzprZXkvbXJrLWM3OWM0Y2Y2Yjk3MDQ2NDFhZDhkOTJmNjI0NGM4MzUwXCJ9LFwiaXZcIjpcIlN2YklWcHJsOTdoejFkbkFcIixcInByb3RlY3RlZFwiOlwiZXlKaGJHY2lPaUpCVjFOZlJVNURYMU5FUzE5Qk1qVTJJaXdpWlc1aklqb2lRVEkxTmtkRFRTSXNJbnBwY0NJNklrUkZSaUo5XCIsXCJ0YWdcIjpcInZ3REU4SVJUamlCbFhoYVNVNk1GNndcIn0iLCJleHAiOjE3MzIxNTUwMDMsImlhdCI6MTczMjE1NDQwMywiaXNzIjoiaHR0cHM6Ly9pZGVudGl0eS5jaWFtLnJlZmluaXRpdi5jb20vYXBpL2lkZW50aXR5L3N0c19wcm9kIiwicnMxIjoiZWE1YzRlNTQyOTRlOTI5MDU3NjU5YTBhNjU5NTkwNTQ0MzUwMTFkOSJ9.SeQsa6CZEoFIP75xB1qZlth5h5pbKSbcLtL7kzHsVsNe3VAgyC4wToVlxF2gUuH0Ew9KlebbpentQh6abwpMCGQsVToOe9lw-y0_IQM29W5yjXDk9V9lK_Kfaz8QNSGwrUlMSOMC29XrXfdfb7EIPJdLnD7rkTHhUEefXBY-SblQ1IK0OsGf9LVeVniaBzw3BTBRNfBupJpIbvBf6IrNNncUVqUkoUXSSqqPh0yvAlE8AucJqI3LZSTEvsjyyaJu5stvMxwhVvANkM4qEdvD8CXeGAxU7guKbgdXuOd-0x-4A9YR8pBmHUvJv9PYs-Ewg2RDeiulhTG7CuWs6Zo8tQ'}
    
	json = {'universe': ['AIRE.S', 'HOLN.S', 'GF.S', 'ROG.S', 'CLN.S', 'ASML.AS', 'SGEF.PA', 'TUI1n.DE', 'DGE.L', 'AMS.S', 'ARYN.S', 'BAYGn.DE', 'NG.L', 'AMA.MC', 'ABI.BR', 'META.O', 'DELL.N', 'AAPL.O', 'AMZN.O', 'CBRE.N', 'CRM.N', 'EL.N', 'FCX.N', 'GOOG.O', 'HAL.N', 'KLAC.O', 'LRCX.O', 'MSFT.O', 'NKE.N', 'NVDA.O', 'TMO.N', 'ULTA.O', 'V.N', 'AVOL.S', 'MONC.MI', 'AEM.N', 'ANGI.O', 'APO.N', 'KKR.N', 'TSLA.O', 'IBKR.O', 'TFG.AS', 'RCP.L', 'RDC.DE', 'QSR.N', 'IAC.O', 'LNSX.F', 'SMHNn.DE', 'DOCM.S', 'ALDA.PA', 'ADYEN.AS', 'SIGNC.S', 'PSH.AS', 'YCA.L', 'F3CG.DE', 'IHPI.L', 'PRX.AS', 'TIMAn.DE', 'CEF.N', 'EXLn.DE', 'GOOGL.O', 'FDEL.PA', 'BYIT.L', 'SPOT.N', 'PPGN.S', 'TNIE.DE', 'APPHG.DE', 'C3RY.DE', 'UMG.AS', 'PANW.O', 'LP68013151', 'LP68617008', 'VNV.ST', 'SYBA.DE', 'RTX.N', 'CRH.N', 'NU.N', 'CPRI.N', 'CAPR.O', 'APP.O', 'PCT.O', 'WGS.O', 'RVNC.O', 'XLS.S', '4X0.DE', 'TSLAX202421500.U', 'TSLAL202424500.U', 'MSFTX202441000.U', 'MSFTL202445000.U', 'METAX202458000.U', 'METAL202462500.U', 'GOOGLX202415500.U', 'GOOGLL202418500.U', 'DELLX202412000.U', 'DELLL202415000.U', 'CRMX202428000.U', 'CRML202432000.U', 'APOX202413500.U', 'APOL202415500.U', 'AMZNX202417500.U', 'AMZNL202420500.U', 'AAPLX202422500.U', 'AAPLL202425500.U', 'ASMO70000L4.E', 'CRMX062433000.U', 'DEA3GPQR.F', 'AZRNW.AS', 'DEA3GRR1.F', 'DEA3GV8V.F', 'MEDZ24', 'FDXZ4', 'STXEZ4', 'ESZ24', 'ES5600X24', 'RR.L', 'LVMH.PA', 'AIRG.DE', 'NOVOb.CO', 'BNRGn.DE', 'ZPLA.S', 'OWL.N', 'IEF.O', 'ILF.P', 'IWM.P', 'STX50EX.DE', 'IBIT.O', 'ZPAL.S', 'LP68060508', 'SXEPEX.DE', 'LP68450281', 'LP68610357', 'RTW.L', 'BE0933899800', '1USZ24', 'MGCZ24', 'GBPEUR=', 'CHFEUR=R', 'USDEUR=R', 'CADEUR=R', 'NOKEUR=R', 'SEKEUR=R', 'DKKEUR=R', 'PLNEUR=R', 'CHF=', 'GBPCHF=', 'CADCHF=', 'NOKCHF=R', 'SGDCHF=R', 'EURCHF=', 'HKDCHF=R', 'JPYCHF=R', 'SEKCHF=R', 'DKKCHF=R', 'AUDCHF=', 'TRYCHF=R', 'PLNCHF=R', 'CHRY.L', 'CRMX062432500.U', 'CRML062434500.U', 'PANWW292439000.U', '.SPX', 'HTGG.DE', 'AZRN.AS', 'SPV2.DE', 'SPXw152456000.U', 'SPXw152455000.U', 'CBREW152412000.U', 'CBREK152413500.U', 'LISP.S', '.GDAXI', '.STOXX50E', '.STOXX', '.STOXXR', 'GBP3MFSR=', 'CIDKK3MD=', 'OINOK3MD=', 'USD3MFSR=', 'EURIBOR3MD=', 'ESc1'], 'fields': ['TR.CLOSEPRICE.DATE', 'TR.BIDPRICE.DATE', 'TR.CLOSEPRICE', 'TR.SETTLEMENTPRICE', 'TR.BIDPRICE', 'TR.ASKPRICE', 'TR.ACCUMULATEDVOLUME', 'TR.VOLUME', 'TR.F.COMSHROUTSTOT', 'TR.COMPANYMARKETCAP', 'TR.BETAWKLY2Y', 'TR.BETADAILY180D', 'TR.OPENINTEREST', 'TR.DELTA', 'TR.IMPLIEDVOLATILITY'], 'parameters': {'SDate': '2024-11-13', 'EDate': '2024-11-20', 'FRQ': 'D'}}
    
[2024-11-21T03:03:06.072648+01:00] - [sessions.platform.rdp.0] - [DEBUG] - [15492 - MainThread] - [http_service] - [request] - HTTP Response id 1
    
	status_code = 504
    
	text = {"error":{"id":"64c1dd1a-1327-4cce-a632-05f19523054b","code":"504","message":"Gateway Timeout","status":"Gateway Timeout"}}

    It looks like to be the timeout on the server side.

  • FFabianB
    FFabianB Newcomer
    https://community.developers.refinitiv.com/discussion/comment/122981#Comment_122981

    @Jirapongse , thanks. So the only option is to split into smaller requests then? Do you have any idea where I can find all parameters that are customizable?

    Is there a writeup available what parameters are available and what the defaults are when using which session?

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @FFabianB

    You can check the C:\<Python>\Lib\site-packages\refinitiv\data\_config_defaults.py file for the configurations.

    Yes, please try to split the request into smaller requests to avoid the server timeout.

    Otherwise, you can try to catch the execption (try/except) and then retry.

Categories