Friday 19 March 2021

flask_login with socketIo makes the flask system breaks every 5-30 refreshes

while I'm working on the front end, somethimes the server breaks and give me this error. all i was doing is modifing the front end and refresh the page to see the results.

not sure what is causing it but it happens every few minutes, i don't think flask is suppose to break with only 1 connection/user is connected to it and bearly using it, on the top of that I'm only hosting it locally.

I'm running a flask app with socket io , flask-sqlalchemy and Postgresql as my database

if __name__ == "__main__":
    socketio.run(app,debug=True )

lunching the app using

note: sqlalchemy drops first , so that could be useful.

python app.py

Exception in thread Thread-425:
Traceback (most recent call last):
  File "<hidden_python_directory>\lib\threading.py", line 954, in _bootstrap_inner
    self.run()
  File "<hidden_python_directory>\lib\threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 680, in _handle_event_internal
    r = server._trigger_event(data[0], namespace, sid, *data[1:])
  File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 704, in _trigger_event
    return self.handlers[namespace][event](*args)
  File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 283, in _handler
    return self._handle_event(handler, message, namespace, sid,
  File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 751, in _handle_event
    ret = handler(*args)
  File "<hidden_applocation>\app.py", line 271, in connect_user
    if current_user.is_authenticated:
  File "<hidden_python_directory>\lib\site-packages\werkzeug\local.py", line 347, in __getattr__
    return getattr(self._get_current_object(), name)
  File "<hidden_python_directory>\lib\site-packages\werkzeug\local.py", line 306, in _get_current_object
    return self.__local()
  File "<hidden_python_directory>\lib\site-packages\flask_login\utils.py", line 26, in <lambda>
    current_user = LocalProxy(lambda: _get_user())
  File "<hidden_python_directory>\lib\site-packages\flask_login\utils.py", line 346, in _get_user
    current_app.login_manager._load_user()
  File "<hidden_python_directory>\lib\site-packages\flask_login\login_manager.py", line 318, in _load_user
    user = self._user_callback(user_id)
  File "<hidden_applocation>\app.py", line 30, in load_user
    return users.query.get(int(id))
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 1018, in get
    return self._get_impl(ident, loading.load_on_pk_identity)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 1135, in _get_impl
    return db_load_fn(self, primary_key_identity)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\loading.py", line 286, in load_on_pk_identity
    return q.one()
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3490, in one
    ret = self.one_or_none()
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3459, in one_or_none
    ret = list(self)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3535, in __iter__
    return self._execute_and_instances(context)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3556, in _execute_and_instances
    conn = self._get_bind_args(
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3571, in _get_bind_args
    return fn(
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\query.py", line 3550, in _connection_from_session
    conn = self.session.connection(**kw)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\session.py", line 1142, in connection
    return self._connection_for_bind(
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\session.py", line 1150, in _connection_for_bind
    return self.transaction._connection_for_bind(
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\orm\session.py", line 433, in _connection_for_bind
    conn = bind._contextual_connect()
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\engine\base.py", line 2302, in _contextual_connect
    self._wrap_pool_connect(self.pool.connect, None),
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\engine\base.py", line 2336, in _wrap_pool_connect
    return fn()
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\base.py", line 364, in connect
    return _ConnectionFairy._checkout(self)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\base.py", line 778, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\base.py", line 495, in checkout
    rec = pool._do_get()
  File "<hidden_python_directory>\lib\site-packages\sqlalchemy\pool\impl.py", line 128, in _do_get
    raise exc.TimeoutError(
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/13/3o7r)

sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/13/3o7r)

then after couple of seconds later it attempts to refresh the page i'm already on but another error keeps showing up.

Traceback (most recent call last):
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 573, in handle_one_response
    result = self.application(self.environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\flask\app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 45, in __call__
    return super(_SocketIOMiddleware, self).__call__(environ,
  File "<hidden_python_directory>\lib\site-packages\engineio\middleware.py", line 60, in __call__
    return self.engineio_app.handle_request(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 571, in handle_request
    return self.eio.handle_request(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\engineio\server.py", line 392, in handle_request
    packets = socket.handle_get_request(
  File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 103, in handle_get_request
    return getattr(self, '_upgrade_' + transport)(environ,
  File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 158, in _upgrade_websocket
    return ws(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\engineio\async_drivers\eventlet.py", line 20, in __call__
    return super(WebSocketWSGI, self).__call__(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 130, in __call__
    self.handler(ws)
  File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 184, in _websocket_handler
    pkt = ws.wait()
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 788, in wait
    for i in self.iterator:
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 643, in _iter_frames
    message = self._recv_frame(message=fragmented_message)
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 669, in _recv_frame
    header = recv(2)
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 578, in _get_bytes
    d = self.socket.recv(numbytes - len(data))
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 365, in recv
    return self._recv_loop(self.fd.recv, b'', bufsize, flags)
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 359, in _recv_loop
    self._read_trampoline()
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 327, in _read_trampoline
    self._trampoline(
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 208, in _trampoline
    return trampoline(fd, read=read, write=write, timeout=timeout,
  File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\__init__.py", line 159, in trampoline
    return hub.switch()
  File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\hub.py", line 313, in switch
    return self.greenlet.switch()
socket.timeout: timed out

127.0.0.1 - - [12/Mar/2021 13:36:55] "GET /socket.io/?EIO=4&transport=websocket&sid=Wu2Ou1x4dEJnfaynAAAo HTTP/1.1" 500 0 30.117126
Traceback (most recent call last):
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 573, in handle_one_response
    result = self.application(self.environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\flask\app.py", line 2464, in __call__
    return self.wsgi_app(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\flask_socketio\__init__.py", line 45, in __call__
    return super(_SocketIOMiddleware, self).__call__(environ,
  File "<hidden_python_directory>\lib\site-packages\engineio\middleware.py", line 60, in __call__
    return self.engineio_app.handle_request(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\socketio\server.py", line 571, in handle_request
    return self.eio.handle_request(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\engineio\server.py", line 392, in handle_request
    packets = socket.handle_get_request(
  File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 103, in handle_get_request
    return getattr(self, '_upgrade_' + transport)(environ,
  File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 158, in _upgrade_websocket
    return ws(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\engineio\async_drivers\eventlet.py", line 20, in __call__
    return super(WebSocketWSGI, self).__call__(environ, start_response)
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 130, in __call__
    self.handler(ws)
  File "<hidden_python_directory>\lib\site-packages\engineio\socket.py", line 184, in _websocket_handler
    pkt = ws.wait()
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 788, in wait
    for i in self.iterator:
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 643, in _iter_frames
    message = self._recv_frame(message=fragmented_message)
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 669, in _recv_frame
    header = recv(2)
  File "<hidden_python_directory>\lib\site-packages\eventlet\websocket.py", line 578, in _get_bytes
    d = self.socket.recv(numbytes - len(data))
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 365, in recv
    return self._recv_loop(self.fd.recv, b'', bufsize, flags)
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 359, in _recv_loop
    self._read_trampoline()
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 327, in _read_trampoline
    self._trampoline(
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 208, in _trampoline
    return trampoline(fd, read=read, write=write, timeout=timeout,
  File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\__init__.py", line 159, in trampoline
    return hub.switch()
  File "<hidden_python_directory>\lib\site-packages\eventlet\hubs\hub.py", line 313, in switch
    return self.greenlet.switch()
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 357, in __init__
    self.handle()
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 390, in handle
    self.handle_one_request()
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 466, in handle_one_request
    self.handle_one_response()
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 616, in handle_one_response
    write(err_body)
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 538, in write
    wfile.flush()
  File "<hidden_python_directory>\lib\socket.py", line 722, in write
    return self._sock.send(b)
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 396, in send
    return self._send_loop(self.fd.send, data, flags)
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 383, in _send_loop
    return send_method(data, *args)
BrokenPipeError: [WinError 10058] A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call

BrokenPipeError: [WinError 10058] A request to send or receive data was disallowed because the socket had already been shut down in that direction with a previous shutdown call

app.py configrations

# for app
app = Flask(__name__)
app.secret_key= #//secret
app.config['WTF_CSRF_SECRET_KEY'] = #//secret

# for database
_string = #//secret
app.config['SQLALCHEMY_DATABASE_URI']= _string
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
db = SQLAlchemy(app)

# for login manager
login = LoginManager(app)
login.init_app(app)

( Update ) I've added this part based on waynetech's answer

#I've added that part
def get_db():
    if 'db' not in g:
        g.db = db.session.connect_to_database()

    return g.db

@app.teardown_appcontext
def teardown_db(exception):
    db = g.pop('db', None)

    if db is not None:
        db.close()

@app.teardown_appcontext
def shutdown_session(exception=None):
print(f'\n now shutting down request. \n')
   db.session.remove()

how I add objects to the database

class1_obj = class1(name=current_user.name ,age=12)
            db.session.add(class1_obj)
            db.session.commit()

(update) now I get this error as well

now shutting down request.

    Traceback (most recent call last):
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 598, in handle_one_response
    write(b''.join(towrite))
  File "<hidden_python_directory>\lib\site-packages\eventlet\wsgi.py", line 537, in write
    wfile.writelines(towrite)
  File "<hidden_python_directory>\lib\socket.py", line 722, in write
    return self._sock.send(b)
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 396, in send
    return self._send_loop(self.fd.send, data, flags)
  File "<hidden_python_directory>\lib\site-packages\eventlet\greenio\base.py", line 383, in _send_loop
    return send_method(data, *args)
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

(update) now I tried removeing the Flask_Login as @waynetech suggested in the comments, it might be doing something to the g element of flask.

so I did disable it but ofcourse I had to remove many other things like all Ajax and most of the SocketIO routs as well.

Then I spamed refresh as fast as i could and the server was stable and never broke!. So it could be casued by the Flask_Login, but I dont know how to go about that.

(Update) everytime I refresh I get the following messages on the power shell console

note :

now shutting down request is showing up due to the print command I have under @app.teardown_appcontext

def shutdown_session(exception=None):

====

127.0.0.1 - - [17/Mar/2021 18:20:54] "GET /static/webfonts/fa-regular-400.woff2 HTTP/1.1" 200 13894 0.006981
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N3U HTTP/1.1" 200 277 0.000997
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N3V HTTP/1.1" 200 277 0.001995
127.0.0.1 - - [17/Mar/2021 11:20:54] "POST /socket.io/?EIO=4&transport=polling&t=NX0_N4P&sid=jxmFQEApPI_OD9OZAAAE HTTP/1.1" 200 219 0.000000
(13996) accepted ('127.0.0.1', 58165)
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N4R&sid=jxmFQEApPI_OD9OZAAAE HTTP/1.1" 200 181 0.001001
(13996) accepted ('127.0.0.1', 58166)
127.0.0.1 - - [17/Mar/2021 11:20:54] "POST /socket.io/?EIO=4&transport=polling&t=NX0_N8d&sid=CF1mhCp5AmERFJGZAAAF HTTP/1.1" 200 219 0.000000
127.0.0.1 - - [17/Mar/2021 11:20:54] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N8e&sid=CF1mhCp5AmERFJGZAAAF HTTP/1.1" 200 181 0.000000
127.0.0.1 - - [17/Mar/2021 11:20:55] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N9N&sid=jxmFQEApPI_OD9OZAAAE HTTP/1.1" 200 181 0.000000
127.0.0.1 - - [17/Mar/2021 11:20:55] "GET /socket.io/?EIO=4&transport=polling&t=NX0_N9S&sid=CF1mhCp5AmERFJGZAAAF HTTP/1.1" 200 181 0.000000

 now shutting down request 

127.0.0.1 - - [17/Mar/2021 18:20:55] "GET /favicon.ico HTTP/1.1" 404 2083 0.013965

 now shutting down request

====

Note: On every refresh, there are few querys and few socketIO's being send to the connected each user

Queries : 1 - pull all new messages. 2 - pull all new notifications.

socketIO: 1- pull the some connection data and update the user profile.

(Update) I FINALLY KNOW WHAT IS CAUSING THIS OVERFLOW

I have a socketIO route that checks if user is authonticated (( using flask_login )) then query data from the database (( using Flask_SQLAlchemy )) and send it to the user

as long as either flask_login or SQLAlchemy involved in one way or another with socketIO , i get that error above

here is an example of what its like.

@socketio.on('custom_emit')
def custom_emit(data):
    if current_user.is_authenticated:
        _id = current_user.get_id()
        x = users.query.filter_by(id=_id).first()
        socketio.emit('update_me',{'my_id':x })


from flask_login with socketIo makes the flask system breaks every 5-30 refreshes

No comments:

Post a Comment