How can files be opened by name in shared storage with native libraries without all files permission?

We are writing a C++ app that is cross compiled for Android. This app uses native libraries (namely gdal and Qt) to read and write files.

These files often have references to each other (e.g. a xml file, referencing other files next to it; or shapefiles, where multiple files have a common base name). The above mentioned libraries need to be able to open files by path (fopen(name)) to get access to all information.

Common use cases are:

• download a zip file using a browser, unzip it using an file manage (e.g. amaze) and open the contents in our app
• use a file sharing app like nextcloud, syncthing, ... to sync a folder. The files in this folder are then edited by our app.

Since API level 30 Google PlayStore restricts file access and only allows unrestricted direct file access in an specific app-specific directory.

In shared folders (e.g. the Download folder, root folders in external storage devices, etc.) only a subset of file types is accessible through direct file access.

The Acess media files from shared storage documentation mentions

If you don't have any storage-related permissions, you can access files in your app-specific directory, as well as media files that are attributed to your app, using the File API.

Media files in our tests include images, videos, ... but not the file types required here (.dbf, .shp, .qgs).

There is also the storage access framework where a DocumentProvider can be used to stream data of individual files and intents like OPEN_DOCUMENT_TREE these will however generate access via URI which is not compatible with native fopen(name) calls.

The only option we found capable of opening files by name (through native labraries) is to use the all files permission (MANAGE_EXTERNAL_STORAGE). This works well but needs to be whitelisted to be published on Google Play Store.

Google support suggests using the system file picker (and sometimes extends the suggestion to "or other privacy friendly approaches").

Q: If your libraries could work with paths before then they still can. Where is it that the problem starts?

A: On Android 11, without a MANAGE_EXTERNAL_STORAGE permission, what we have seen in our multiple tests is that directly opening non-media files in shared locations via their path strings is not working anymore. The file opening fails. We have also noticed that doing a directory listing (using Qt APIs) only shows media file extensions, skipping all other types.

How is it possible to open files by file path (string) in shared locations with native libraries without the all files permission (MANAGE_EXTERNAL_STORAGE)?

from How can files be opened by name in shared storage with native libraries without all files permission?