My flask_socketio api is very long in load #1628

MathiasDrapier · 2021-07-07T07:03:24Z

MathiasDrapier
Jul 7, 2021

Hello,
I have noticed that when I test my api with jmeter and send more than 20 requests per second the response time starts to increase exponentially.
Even for a request that only returns "hello".
I tested on a not very powerful machine (2 giga of ram and 2 core) and also on a very powerful machine (32 giga and 12 core) and the result is almost the same

Here are the results of the werkzeug profiler :

    |          650 function calls in 8.875 seconds
    |
    |    Ordered by: internal time, call count
    |
    |    ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    |         1    8.874    8.874    8.874    8.874 {method 'enable' of '_lsprof.Profiler' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/http/server.py:269(parse_request)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/semaphore.py:70(acquire)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:658(get_environ)
    |         9    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:462(get)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/socket.py:302(makefile)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:218(_parsegen)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:101(push)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:471(_parse_headers)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/http/client.py:204(_read_headers)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/engineio/middleware.py:41(__call__)
    |        12    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:894(match)
    |        15    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1603(bind_to_environ)
    |         2    0.000    0.000    8.875    4.437 {method 'switch' of 'greenlet.greenlet' objects}
    |         3    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:414(__get__)
    |         8    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:150(__getattr__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1844(match)
    |         4    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:29(_splitparam)
    |         4    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:564(get_content_type)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:404(handle_one_request)
    |         1    0.000    0.000    8.874    8.874 /usr/local/lib/python3.9/site-packages/flask/app.py:2083(__call__)
    |         4    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/_internal.py:150(_wsgi_decoding_dance)
    |         1    0.000    0.000    0.000    0.000 {method 'getsockname' of '_socket.socket' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/corolocal.py:27(_patch)
    |         6    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/_policybase.py:293(header_source_parse)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/wrappers/request.py:107(__init__)
    |        54    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
    |         7    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:316(readline)
    |         6    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:241(top)
    |         9    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:78(readline)
    |         1    0.000    0.000    0.000    0.000 {method 'recv_into' of '_socket.socket' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/ctx.py:372(push)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/http/client.py:222(parse_headers)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:157(__setattr__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/sansio/request.py:120(__init__)
    |         1    0.000    0.000    8.874    8.874 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:463(handle_one_response)
    |         8    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/_policybase.py:281(_sanitize_header)
    |         9    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:128(__next__)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/flask/app.py:1501(full_dispatch_request)
    |         8    0.000    0.000    0.000    0.000 {method 'readline' of '_io.BufferedReader' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:350(setup)
    |         1    0.000    0.000    0.000    0.000 {method 'setsockopt' of '_socket.socket' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/parser.py:41(parse)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/app.py:1832(preprocess_request)
    |         8    0.000    0.000    0.000    0.000 {method 'match' of 're.Pattern' objects}
    |         8    0.000    0.000    0.000    0.000 {method 'upper' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/parser.py:59(parsestr)
    |        36    0.000    0.000    0.000    0.000 {built-in method builtins.len}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:221(push)
    |        10    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
    |         8    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/utils.py:51(_has_surrogates)
    |         2    0.000    0.000    0.000    0.000 {method 'readlines' of '_io._IOBase' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:139(__init__)
    |         8    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/_policybase.py:311(header_fetch_parse)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/socket.py:690(readinto)
    |         3    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:538(_get_current_object)
    |        12    0.000    0.000    0.000    0.000 {method 'search' of 're.Pattern' objects}
    |         1    0.000    0.000    8.874    8.874 /usr/local/lib/python3.9/site-packages/werkzeug/middleware/profiler.py:93(__call__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1738(__init__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/ctx.py:296(__init__)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/flask_socketio/__init__.py:42(__call__)
    |        14    0.000    0.000    0.000    0.000 {method 'encode' of 'str' objects}
    |         3    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:588(get_content_maintype)
    |        23    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/socket.py:678(__init__)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/app.py:1748(create_url_adapter)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:178(_call_parse)
    |        23    0.000    0.000    0.000    0.000 {built-in method builtins.isinstance}
    |         5    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/random.py:237(_randbelow_with_getrandbits)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/greenio/base.py:308(makefile)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/flask/app.py:1477(dispatch_request)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/inspect.py:172(_has_code_flag)
    |         3    0.000    0.000    0.000    0.000 {method 'copy' of 'dict' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:831(<lambda>)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/wrappers.py:76(blueprint)
    |         1    0.000    0.000    8.875    8.875 /usr/src/app/src/db.py:36(getConn)
    |         6    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:479(set_raw)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:53(__init__)
    |         7    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:701(<genexpr>)
    |        14    0.000    0.000    0.000    0.000 {method 'strip' of 'str' objects}
    |        17    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:184(close)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:70(close)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/_internal.py:165(_get_environ)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1539(bind)
    |         3    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/globals.py:30(_lookup_req_object)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/wsgi.py:75(_get_server)
    |         6    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/_internal.py:131(_to_str)
    |         9    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/ctx.py:237(push)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/werkzeug/middleware/profiler.py:102(runapp)
    |         1    0.000    0.000    8.875    8.875 /usr/src/app/src/modules/consumption/router.py:64(postMbusConsumption)
    |         8    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:120(__init__)
    |         1    0.000    0.000    8.874    8.874 /usr/local/lib/python3.9/cProfile.py:106(runcall)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/ctx.py:362(match_request)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/flask/app.py:2040(wsgi_app)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:210(_pop_message)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:122(pushlines)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:385(_read_request_line)
    |         6    0.000    0.000    0.000    0.000 {method 'count' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:789(get_environ)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:78(__init__)
    |         8    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:197(_new_message)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:339(__init__)
    |         6    0.000    0.000    0.000    0.000 {method 'decode' of 'bytes' objects}
    |         3    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1691(_get_wsgi_string)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:806(process_request)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/sessions.py:70(__init__)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/wrappers.py:95(blueprints)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/ctx.py:228(__init__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/greenio/base.py:335(_recv_loop)
    |         8    0.000    0.000    0.000    0.000 {method 'lstrip' of 'str' objects}
    |         4    0.000    0.000    0.000    0.000 {method 'partition' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/random.py:290(randrange)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/wsgi.py:92(get_host)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/sansio/utils.py:56(get_host)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/weakref.py:444(__contains__)
    |         2    0.000    0.000    0.000    0.000 {method 'set' of 'ContextVar' objects}
    |         6    0.000    0.000    0.000    0.000 {method 'find' of 'str' objects}
    |        10    0.000    0.000    0.000    0.000 {method 'get' of 'ContextVar' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/greenio/base.py:372(recv_into)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:54(addr_to_host_port)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:173(feed)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/sessions.py:335(get_signing_serializer)
    |         1    0.000    0.000    0.000    0.000 {method 'write' of '_io.StringIO' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/sessions.py:150(make_null_session)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/datastructures.py:2224(__init__)
    |         4    0.000    0.000    0.000    0.000 {method 'endswith' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/app.py:1968(request_context)
    |         6    0.000    0.000    0.000    0.000 {method 'replace' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/greenio/base.py:190(__getattr__)
    |         3    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/local.py:400(bind_f)
    |         1    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/corolocal.py:47(__setattr__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1064(build)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/typing.py:269(inner)
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/hubs/timer.py:54(__call__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/random.py:334(randint)
    |         2    0.000    0.000    0.000    0.000 {method 'read' of '_io.StringIO' objects}
    |         7    0.000    0.000    0.000    0.000 {method 'popleft' of 'collections.deque' objects}
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:375(handle)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/urllib/parse.py:644(unquote)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/weakref.py:417(__setitem__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/app.py:1586(ensure_sync)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:2086(get_default_redirect)
    |         2    0.000    0.000    0.000    0.000 {method 'extend' of 'collections.deque' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/message.py:303(set_payload)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/weakref.py:404(__getitem__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/hubs/__init__.py:101(get_hub)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/socket.py:729(readable)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/inspect.py:81(ismethod)
    |         3    0.000    0.000    0.000    0.000 {built-in method greenlet._greenlet.getcurrent}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/inspect.py:190(iscoroutinefunction)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/sessions.py:350(open_session)
    |         1    0.000    0.000    0.000    0.000 {method 'join' of 'bytes' objects}
    |         1    0.000    0.000    0.000    0.000 {built-in method _heapq.heappop}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/greenio/base.py:180(_set_io_refs)
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/greenio/base.py:177(_get_io_refs)
    |         2    0.000    0.000    0.000    0.000 {built-in method time.time}
    |         1    0.000    0.000    8.875    8.875 /usr/local/lib/python3.9/site-packages/eventlet/greenthread.py:219(main)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/_internal.py:476(_encode_idna)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/app.py:1947(app_context)
    |         4    0.000    0.000    0.000    0.000 {method 'seek' of '_io.StringIO' objects}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/wrappers.py:61(endpoint)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/config.py:16(__get__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/parser.py:17(__init__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/hubs/hub.py:286(ensure_greenlet)
    |         2    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
    |         2    0.000    0.000    0.000    0.000 {method 'truncate' of '_io.StringIO' objects}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/email/feedparser.py:125(__iter__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/wsgi.py:312(__init__)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/inspect.py:159(isfunction)
    |         1    0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x7f3abdc99980}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/datastructures.py:1373(__init__)
    |         2    0.000    0.000    0.000    0.000 {method 'copy' of 'list' objects}
    |         1    0.000    0.000    0.000    0.000 {method 'getrandbits' of '_random.Random' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/functools.py:420(_unwrap_partial)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/socket.py:736(writable)
    |         1    0.000    0.000    0.000    0.000 {method '_checkReadable' of '_io._IOBase' objects}
    |         2    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/signals.py:25(send)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/werkzeug/routing.py:1711(update)
    |         1    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
    |         1    0.000    0.000    0.000    0.000 {method '_checkClosed' of '_io._IOBase' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/eventlet/support/__init__.py:36(clear_sys_exc_info)
    |         1    0.000    0.000    0.000    0.000 {method 'bit_length' of 'int' objects}
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/site-packages/flask/app.py:1547(try_trigger_before_first_request_functions)
    |         1    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
    |         1    0.000    0.000    0.000    0.000 {method 'append' of 'collections.deque' objects}
    |         1    0.000    0.000    0.000    0.000 {method 'groupdict' of 're.Match' objects}
    |         1    0.000    0.000    0.000    0.000 <werkzeug routing>:1(<builder:'/consumption/mbusConsumption'>)
    |         1    0.000    0.000    0.000    0.000 /usr/local/lib/python3.9/typing.py:1340(cast)

miguelgrinberg · 2021-07-07T08:25:53Z

miguelgrinberg
Jul 7, 2021
Maintainer

You are using eventlet, which is probably confusing your profiler. I suggest that for profiling you use the threading mode, which is fully compatible with the werkzeug profiler. Can I ask you to repeat the test that way?

0 replies

MathiasDrapier · 2021-07-07T09:23:35Z

MathiasDrapier
Jul 7, 2021
Author

Thanks this help me find the issue.
My program takes a lot of time in the acquire function of a semaphore that I use for db connection.
I create several connections to the database (50) that I protect with a semaphore so that the thread does not use it at the same time.
I don't understand why the blocking time is so long. And I get the same with 2 or 50 or even 100 connections.
I know the problem is not socketio dependent but I would need some help if you have the time.

Here is my code to manage the connections to the database :

    def __init__(self):
        for i in range(0, 50):
            conn = psycopg2.connect(user=dbUser,
                                    password=dbPassword,
                                    host=dbHost,
                                    port=dbPort,
                                    database=dbDatabase
                                    )
            conn.autocommit = True
            self.conns.append(conn)
            sem = RLock()
            self.sems.append(sem)

    def getConn(self):
        index = random.randint(0, 49)
        self.sems[index].acquire()
        conn = self.conns[index]
        return conn, cursor

    def putConn(self, conn, cursor):
        cursor.close()
        index = self.conns.index(conn)
        self.sems[index].release()

When I need a connection I call getConn and then I use it I call putConn

2 replies

miguelgrinberg Jul 7, 2021
Maintainer

I don't really see anything that could cause what you describe in the snippet of code you pasted here.

MathiasDrapier Jul 7, 2021
Author

Ok thanks for the help regarding the profiler.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

My flask_socketio api is very long in load #1628

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 2 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Uh oh!

My flask_socketio api is very long in load #1628

Uh oh!

MathiasDrapier Jul 7, 2021

Replies: 2 comments · 2 replies

Uh oh!

miguelgrinberg Jul 7, 2021 Maintainer

Uh oh!

MathiasDrapier Jul 7, 2021 Author

Uh oh!

miguelgrinberg Jul 7, 2021 Maintainer

Uh oh!

MathiasDrapier Jul 7, 2021 Author

MathiasDrapier
Jul 7, 2021

Replies: 2 comments 2 replies

miguelgrinberg
Jul 7, 2021
Maintainer

MathiasDrapier
Jul 7, 2021
Author

miguelgrinberg Jul 7, 2021
Maintainer

MathiasDrapier Jul 7, 2021
Author