Thursday, 20 June 2019

pipe/stream gnupg output to tarfile in

I have the following code but obviously this is not real streaming. It is the best I could find but it reads the whole input file into memory first. I want to stream it to tarfile module without using all my memory when decrypting huge (>100Gb files)

import tarfile, gnupg                                                                                                                                                                                                                                
gpg = gnupg.GPG(gnupghome='C:/Users/niels/.gnupg')                                                                         

with open('103330-013.tar.gpg', 'r') as input_file:                                                                                                                                                                                                   
    decrypted_data = gpg.decrypt(input_file.read(), passphrase='aaa')                                                       
    # decrypted_data.data contains the data                                                                                 
    decrypted_stream = io.BytesIO(decrypted_data.data)                                                                      

    tar = tarfile.open(decrypted_stream, mode='r|')                                                                                                                                                                                                 
    tar.extractall()                                                                                                                                                                                                                                
    tar.close()



from pipe/stream gnupg output to tarfile in

No comments:

Post a Comment