Wednesday, 7 October 2020

Open multiple pickle files from Jupyter notebook folder doesn't work

I'm using jupyter notebook on a server (the folders are not on my computer). I have folder with 30 dataframes pickled that have exactly the same columns. They are all saved in the next path:

Reut/folder_no_one/here_the_files_located

I want to open them all and to concat them. I know I could do something like this:

df1=pd.read_pickle('table1')
df2=pd.read_pickle('table2')
df3=pd.read_pickle('table3')
...
#and then concat

but i'm sure there is better and smarters way to do so. I have tried to open all the files and save them seperatly as following:

num=list(range(1, 33)) #number of tables I have in the folder
path_to_files=r'Reut/here_the_files_located'
Path=r'Reut/folder_no_one/here_the_files_located'

{f"df{num}" : pd.read_pickle(file) for num, file in enumerate(Path(path_to_files).glob('*.pickle'))}

but I get error with this:

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) in ----> 1 {f"df{num}" : pd.read_pickle(file) for num, file in enumerate(Path(path_to_files).glob('*.pickle'))}

TypeError: 'str' object is not callable

I have tried to play and o put different version of the path, also not to put path (because my notebook is where those files are) but I keep getting the same error.

*it's important to mention that when I can open those files without specifiying the path when the notebook is inside that folder as well.

My end goal is to open and concat all those tables as one big table automatically.



from Open multiple pickle files from Jupyter notebook folder doesn't work

No comments:

Post a Comment