Calling Firestore Database from another Cloud Project shows "Permission Error"

After reading every piece of question, documentation available online, we couldn't find a solution, so posting a question here.

Our Setup :

1. Project FB: Used only for Firestore in Dev mode. We want to access data from this project in our dev backend server, hosted on a different cloud project.

2. Project GCP: A GCP Project with app deployed in GAE that makes a simple get() call to Project FB's document. The app is in Python, and it works perfectly in localhost, but not after it's deployed.

Sample Flask code that we are testing with:

route('/test-fb', methods=['POST', 'GET'])
def test():
    doc = doc_ref.get()
    if doc.exists:
        print(u'Document data: {}'.format(doc.to_dict()))
    else:
        print(u'No such document!')
    return make_response("Firestore worked!")

Note:

1. This is not a question on Firestore Rules, because we are using service account key. To be 100% sure in any case, we have rules set to: true always

   match /{document=**} { allow read, write: if true

2. We generate the Service Account private key by going to "Settings -> Service Accounts", generate new private key. Then use the Admin SDK config snippet code in Python above our code. This works perfectly in localhost.

3. Aware of the Service Account permissions needed, we added many permissions including 'Editor', 'Storage Admin' and 'Cloud Datastore Owner' to the Project FB IAM account for GAE service Account of Project GCP (@appspot.gserviceaccount.com)

4. All the Firestore packages, any other dependency are updated to the latest version.

5. Created new keys to test again. For Project FB, updated Credentials -> Key restrictions and set them to unrestricted so any domains can access them.

6. Deleted the versions, and tried again many times at different times of the day as well. The deployments happen through triggered cloud builds in Project GCP. The cloud builds are successful. Also, all routes function perfectly except the one in which we are reading the Firestore document (code above).

7. Deleted cookies, and tried different browsers.

8. Instead of using the snippet code, also tried the google-cloud-firestore package: https://pypi.org/project/google-cloud-firestore/

9. Both the projects are in the same location (US multilocation)

Please advise on what we could be doing wrong, and what else we can try? We are lost at this point, and this simple task has taken us several days, and we've tried all permutations of above steps multiple times to double check.

GAE Response on a request to the server:

from Calling Firestore Database from another Cloud Project shows "Permission Error"