Sunday 8 August 2021

Can you pipe non-text data from a python script via PowerShell?

I am fairly familiar with bash and know how to some basic scripting stuff involving pipes, and use them as a 'back end' to run python scripts in sequence.

However, for a new project I've been tasked with I can only use PowerShell. I've found that I can rewrite my previous shell scripts fine, buta I hear that you can pipe non-text data in PowerShell too.

My question is:

Is it possible to pipe non-text output (primarily a pandas dataframe) from a python script into another python script via PowerShell?

Something similar to:

script1.py | script2.py

If so, what are the logistics in regards to the python script? I.E can you still return to sys.stdout?

EDIT:

To better explain to usecase to be in line with the comments I've received.

I have two python scripts, test1.py:

#test1.py
import pandas as pd
import sys


def main():
    columns = ['A', 'B', 'C']
    data = [
        ['hello', 0,  3.14],
        ['world', 1,  2.71],
        ['foo',   2,  0.577],
        ['bar',   3,  1.61]

    ]

    df = pd.DataFrame(data, columns=columns)
    return df


if __name__ == "__main__":
    main().to_csv(sys.stdout, index_label=False)

and test2.py:

#test2.py
import pandas as pd
import sys


def main():
    df = pd.read_csv(sys.stdin)
    print(df.dtypes)


if __name__ == "__main__":
    main()

I'm using PowerShell to do some automation, and need to pipe the output of one script to the other; python test1.py | python test2.py works perfectly fine.

My question is, I have heard that you can pipe non-text data in PowerShell, which you can't do in Bash (I think), so is it possible to pipe the Dataframe as it is? (without having to convert to a CSV or some other string encoding)



from Can you pipe non-text data from a python script via PowerShell?

No comments:

Post a Comment