Access EDGAR Filings in python

Miriam
Miriam Newcomer
edited May 26 in Refinitiv Data Platform

Hi, I've been trying to retrieve filings from a list of different RICs and Dates (specific range for each RIC) but im having difficulties using the code examples available. Would really appreciate help to code a request to access the MD&A Section of 10-Q/10-K of many companies for which i have the RICs and dates. I've been trying the following code but I believe there is an error with the url in: request_definition = endpoint_request.Definition(
method = rd.delivery.endpoint_request.RequestMethod.POST,
url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup',
body_parameters = body
https://github.com/LSEG-API-Samples/Article.RD.Python.PredictiveModellingForFilings/blob/main/Filings.ipynb


Also, all examples found refer to previous verision of library (refinitiv, not lseg)

Thanks!

Welcome!

It looks like you're new here. Sign in or register to get started.

Answers

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @Miriam

    Thank you for reaching out to us.

    Please share what the error is.

    You may try the following code to check for the error.

    body = {
        "from": [
            {
                "identifierTypes": ["RIC"],
                "values": ["LSEG.L"]
            }],
        "to": [
            {"identifierTypes": ["PermID"],
             "objectTypes": ["organization"]
            }],
        "type": "auto"
    }
    request_definition = endpoint_request.Definition(
                method = rd.delivery.endpoint_request.RequestMethod.POST,
                url = 'https://api.refinitiv.com/discovery/symbology/v1/lookup',
                body_parameters = body
            )
    
    response = request_definition.get_data()
    print(response)
  • Miriam
    Miriam Newcomer
    edited May 26

    TimeoutError: timed out

    Traceback (most recent call last):
    File "<ipython-input-9-b5f7a8f3bcd0>", line 1, in <module>
    permid = lookup.convert()
    File "/usr/local/lib/python3.9/dist-packages/refinitiv/data/discovery/symbology/lookup.py", line 221, in convert
    result = self._send_request()
    File "/usr/local/lib/python3.9/dist-packages/refinitiv/data/discovery/symbology/lookup.py", line 280, in _send_request
    response = self._api_client.call_api(url, method="POST", body=body)
    File "/usr/local/lib/python3.9/dist-packages/refinitiv/data/core/_platform/api_client.py", line 361, in call_api
    response = self.request(method, url, **kwargs)
    File "/usr/local/lib/python3.9/dist-packages/refinitiv/data/core/_platform/api_client.py", line 404, in request
    response = self._session.request(method, url, **kwargs)
    File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
    File "/usr/local/lib/python3.9/dist-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
    File "/usr/local/lib/python3.9/dist-packages/requests/adapters.py", line 530, in send
    raise Timeout(e, request=request)
    requests.exceptions.Timeout: timed out

    Some codes have worked but return table full of NaN, do you have an example code that works to retrieve the management section of 10-Q filings as text?

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @Miriam

    It looks like to be a HTTP timeout. You may need to increase the timeout value by running the following code before opening a session.

    config = rd.get_config()
    config.set_param("logs.transports.file.enabled", True)
    config.set_param("logs.transports.file.name", "lseg-data-lib.log")
    config.set_param("logs.level", "debug")
    config.set_param("http.request-timeout", 500)

    The code is also enable the debug log. The lseg-data-lib.log file will be created.

    If it doesn't help, please remove any private information from the log file and then share it when the problem occurred.

  • Miriam
    Miriam Newcomer

    This is the full code. With the code you provided it doesn't return an error but it returns no output as can be seen in the end. I used exact code as in the article, except I updated for lseg.data instead of refinitiv (i tried both anyways)

    Thanks for helping

    https://community.developers.refinitiv.com/home/leaving?allowTrusted=1&target=https%3A%2F%2Fgithub.com%2FLSEG-API-Samples%2FArticle.RD.Python.PredictiveModellingForFilings%2Fblob%2Fmain%2FFilings.ipynb

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @Miriam

    I can run the following code properly with the platform.ldp session which requires an username, password and application key.

        "sessions": {
            "default": "platform.ldp",
            "platform": {
                "ldp": {
                    "app-key": "<appkey>",
                    "username": "<username>",
                    "password": "<password>",
                    "signon_control":true
                },

    The code is:

    # Use the RIC to define our company identifier
    identifier = "IBM.N"    
    
    # Data ranges to capture our data sets used to predict sentiment
    start = "2020-01-01T00:00:00Z"
    end = "2024-10-31T23:59:59Z"
    print(SymbolLookup(ric=identifier).convert())
    response = filings.search.Definition(
        query = FilingsQuery().form_type("10-Q")
                              .orgid(SymbolLookup(ric=identifier).convert())
                              .feed(FilingsQuery.Feed.EDGAR)
                              .sections(["ManagementDiscussion"])
                              .start_date(start)
                              .end_date(end)
                              .limit(100)
                              .graphQL
    ).get_data()
    image.png

    To investigate the issue, please share the debug log file.

  • Miriam
    Miriam Newcomer

    Thanks, I've tried with platform session. However, when I reach that part it displays the following error, eventhough session is open

    :

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-50-e69ab03c8c48> in <cell line: 0>()      5 start = "2020-01-01T00:00:00Z"      6 end = "2024-10-31T23:59:59Z"----> 7 print(SymbolLookup(ric=identifier).convert())      8 response = filings.search.Definition(      9     query = FilingsQuery().form_type("10-Q")

    3 frames

    /usr/local/lib/python3.11/dist-packages/lseg/data/_core/session/tools.py in raise_if_closed(session)     34         error_message = "Session is not opened. Can't send any request"     35         session.error(error_message)---> 36         raise ValueError(error_message)     37      38 
    ValueError: Session is not opened. Can't send any request
  • Jirapongse
    Jirapongse ✭✭✭✭✭

    The code to open a session should look like this:

    config = ld.get_config()
    config.set_param("logs.transports.file.enabled", True)
    config.set_param("logs.transports.file.name", "lseg-data-lib.log")
    config.set_param("logs.level", "debug")
    config.set_param("http.request-timeout", 500)
    
    session = ld.session.platform.Definition(
        app_key = "<appkey>",
        grant = ld.session.platform.GrantPassword(
        username = "<username>",
        password = "<password>",
        ), signon_control=True
        ).get_session()
    session.open()
    
    ld.session.set_default(session)

    Please share the lseg-data-lib.log file if the code doesn't work.

  • Miriam
    Miriam Newcomer

    I attached the file, as it keeps retunrning same error. Thanks!

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    From the log file, you are using the desktop session (http://localhost:9000).

    Do you have a LD library configuration file (lseg-data.config.json) in the current working directory?

  • Miriam
    Miriam Newcomer

    I just have the file attached in previous message and the following

    But I haven't opened Lseg Workspace app in desktop

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    It looks better. However, the takeExclusiveSignOnControl parameter in the request is false.

    Its value should be true according to the signon_control parameter in the code.

    session = ld.session.platform.Definition(
    app_key = "<appkey>",
    grant = ld.session.platform.GrantPassword(
    username = "<username>",
    password = "<password>",
    ), signon_control=True
    ).get_session()
    session.open()

    What version of LSEG Data Library are you using?

    image.png
  • Miriam
    Miriam Newcomer
    edited May 27

    Now I managed to run the code! But I got this output..

    image.png

    In raw it appears:

    'extensions': {'warnings': [{'field': 'FinancialFiling.FilingDocument', 'message': 'User did not have entitlements to the field.', 'type': 'NO_ENTITLEMENTS_FOR_FIELD', 'warningCode': 'DW003'}]}})
  • Jirapongse
    Jirapongse ✭✭✭✭✭

    @Miriam

    It looks like to be a permission issue.

    Please contact your LSEG account team or sales team directly to verify your permission.

    Otherwise, please share the debug log file so I can verify the request and response messages.

  • Miriam
    Miriam Newcomer

    Thanks a lot! I've been trying to contact my account manager but if you could check it, I would really appreciate it.

  • Jirapongse
    Jirapongse ✭✭✭✭✭

    Yes, it should be a permission issue.

    I used the same query and could retrieve the data properly.

Welcome!

It looks like you're new here. Sign in or register to get started.

Welcome!

It looks like you're new here. Sign in or register to get started.