Friday, 16 October 2020

How to use blobs > 1GB in zodb relstorage with sqlite backend?

I am trying to save a blob > 1GB via relstorage in a sqlite backend. The following minimum working example

  • removes any previously created databases along with the blob directory,
  • create a new database and
  • saves a blob with 1GB to the database.

Minimum Working Example:

import os
import shutil
from ZODB import blob, config

connectionString = """
%import relstorage
<zodb main>
<relstorage>
    blob-dir blob
    keep-history false
    cache-local-mb 0
    <sqlite3>
        data-dir .
    </sqlite3>
</relstorage>
</zodb>
"""

# cleaning up
for x in os.listdir():
    if "sqlite" in x:
        os.remove(x)

shutil.rmtree("blob", True)

# creating database
db = config.databaseFromString(connectionString)

with db.transaction() as conn:
    conn.root.blob = blob.Blob()
    with conn.root.blob.open("w") as f:
        f.write(b"\0" * 1024 ** 3)

During the commit (at the __exit__ method of the db.transaction with-block) of the newly created object an error occurs with the following final exception:

sqlite3.InterfaceError: Error binding parameter 2 - probably unsupported type.

Saving a blob with size 1024 ** 2 does not raise the exception.

How is it possible to save a blob in a sqlite backend using relstorage without the usage of a shared blob directory?



from How to use blobs > 1GB in zodb relstorage with sqlite backend?

No comments:

Post a Comment