Monday, 25 January 2021

Issues with Firebase Admin SDK on elastic beanstalk

Just to give a brief introduction, I am using elastic beanstalk from Amazon to deploy my python (Falcon) application. I have two environments of elastic beanstalk (one being the main app server and the other one a worker environment). The worker environment receives HTTP requests for async tasks and then executes them. Most of the tasks are either push notifications (mobile devices) or sending emails. The strange part is that my tasks are failing with errors that I am unable to determine the cause for (this works fine for my local environment). Some of the tracebacks look like this:

self.send_message(message=message)
File "/var/app/current/application/services/push.py", line 114, in send_message
response = messaging.send_multicast(message)
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/firebase_admin/messaging.py", line 167, in send_multicast
return _get_messaging_service(app).send_all(messages, dry_run)
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/firebase_admin/messaging.py", line 390, in send_all
batch.execute()
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/http.py", line 1540, in execute
self._execute(http, self._order, self._requests)
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/http.py", line 1470, in _execute
self._batch_uri, method="POST", body=body, headers=headers
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/google_auth_httplib2.py", line 201, in request
uri, method, body=body, headers=request_headers, **kwargs)
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
cachekey,
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request
conn, request_uri, method, body, headers
File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1589, in _conn_request
response = conn.getresponse()
File "/usr/lib64/python3.7/http/client.py", line 1369, in getresponse
response.begin()
File "/usr/lib64/python3.7/http/client.py", line 310, in begin
version, status, reason = self._read_status()
File "/usr/lib64/python3.7/http/client.py", line 271, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib64/python3.7/socket.py", line 589, in readinto
return self._sock.recv_into(b)
File "/usr/lib64/python3.7/ssl.py", line 1071, in recv_into
return self.read(nbytes, buffer)
File "/usr/lib64/python3.7/ssl.py", line 929, in read
return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out
    File "/var/app/current/application/services/push.py", line 152, in execute_push
    self.send_message(message=message)
    File "/var/app/current/application/services/push.py", line 114, in send_message
    response = messaging.send_multicast(message)
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/firebase_admin/messaging.py", line 167, in send_multicast
    return _get_messaging_service(app).send_all(messages, dry_run)
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/firebase_admin/messaging.py", line 390, in send_all
    batch.execute()
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
    return wrapped(*args, **kwargs)
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/http.py", line 1540, in execute
    self._execute(http, self._order, self._requests)
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/http.py", line 1470, in _execute
    self._batch_uri, method="POST", body=body, headers=headers
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/google_auth_httplib2.py", line 201, in request
    uri, method, body=body, headers=request_headers, **kwargs)
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
    cachekey,
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request
    conn, request_uri, method, body, headers
    File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1589, in _conn_request
    response = conn.getresponse()
    File "/usr/lib64/python3.7/http/client.py", line 1369, in getresponse
    response.begin()
    File "/usr/lib64/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
    File "/usr/lib64/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
    File "/usr/lib64/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
    File "/usr/lib64/python3.7/ssl.py", line 1071, in recv_into
    return self.read(nbytes, buffer)
    File "/usr/lib64/python3.7/ssl.py", line 929, in read
    return self._sslobj.read(len, buffer)
    ssl.SSLError: [SSL: UNKNOWN_STATE] unknown state (_ssl.c:2570)
    
      File "/var/app/current/application/services/push.py", line 114, in send_message
      response = messaging.send_multicast(message)
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/firebase_admin/messaging.py", line 167, in send_multicast
      return _get_messaging_service(app).send_all(messages, dry_run)
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/firebase_admin/messaging.py", line 390, in send_all
      batch.execute()
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
      return wrapped(*args, **kwargs)
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/http.py", line 1540, in execute
      self._execute(http, self._order, self._requests)
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/googleapiclient/http.py", line 1470, in _execute
      self._batch_uri, method="POST", body=body, headers=headers
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/google_auth_httplib2.py", line 201, in request
      uri, method, body=body, headers=request_headers, **kwargs)
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1994, in request
      cachekey,
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1651, in _request
      conn, request_uri, method, body, headers
      File "/var/app/venv/staging-LQM1lest/lib/python3.7/site-packages/httplib2/__init__.py", line 1618, in _conn_request
      content = response.read()
      File "/usr/lib64/python3.7/http/client.py", line 468, in read
      return self._readall_chunked()
      File "/usr/lib64/python3.7/http/client.py", line 575, in _readall_chunked
      chunk_left = self._get_chunk_left()
      File "/usr/lib64/python3.7/http/client.py", line 563, in _get_chunk_left
      self._read_and_discard_trailer()
      File "/usr/lib64/python3.7/http/client.py", line 536, in _read_and_discard_trailer
      line = self.fp.readline(_MAXLINE + 1)
      AttributeError: 'NoneType' object has no attribute 'readline'
      

      Some other exceptions are as follows:

      ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:2570)
      ssl.SSLError: [SSL: DECRYPTION_FAILED_OR_BAD_RECORD_MAC] decryption failed or bad record mac (_ssl.c:2570)
      ssl.SSLError: [SSL: BLOCK_CIPHER_PAD_IS_WRONG] block cipher pad is wrong (_ssl.c:2570)
      

      The code looks like this:

       message = messaging.MulticastMessage(
                  tokens=tokens,
                  data=data,
                  notification=messaging.Notification(
                      title=title,
                      body=body
                  )
              )
      response = messaging.send_multicast(message)
      

      This is quite strange for me since the tasks are failing and as a result, retrying. But somehow I am receiving multiple notifications. That means the notifications are being sent but the tasks are getting failed. Please help me out in understanding what could be the problem. Does it have something to do with the system date time?



      from Issues with Firebase Admin SDK on elastic beanstalk

      No comments:

      Post a Comment