I somehow got into a weird situation, and this was working only a few hours ago. I have a Python script that works just fine, i.e. I can see that it's doing what it's supposed to do (creating files, starting a process, etc.). When I run this script through CMD or PowerShell, I can see the output just fine, but for some reason I have no output when I run it through Azure Batch.
Sample code (script called through Batch):
if __name__ == "__main__":
import time
start = time.time()
print(f"Starting job at {datetime.fromtimestamp(start)}")
There are at least a dozen more print statements throughout the script.
python can be resolved through the PATH env var, and it's under C:\Program Files (x86)\Python37-32. Python 3.9 is also installed through Anaconda3, if that makes a difference.
This was originally called through Azure Batch with:
cmd /C C:\ProgramData\Anaconda3\Scripts\activate.bat &&
robocopy C:\SomeFolder . /e /NFL /NDL /NJH /NJS /nc /ns /np & if not errorlevel 4 ver && .\unzip -o some-id.zip &&
"C:\Program Files (x86)\Python37-32\python.exe" .\entry-script.py &&
"C:\Program Files (x86)\Python37-32\python.exe" .\upload.py some-id
This used to work. Since it stopped working, I've tried different combinations and tried to get rid of cmd /C but robocopy fails then. I was able to simplify it to:
robocopy C:\SomeFolder . /e /NFL /NDL /NJH /NJS /nc /ns /np & if not errorlevel 4 ver && .\unzip -o some-id.zip &&
python.exe .\entry-script.py &&
python.exe .\upload.py some-id
Line breaks are provided here only to make the text readable, they don't exist in the real command. This still does not provide any output. I also redirected the output for the script with python.exe .\entry-script.py > stdout.txt 2> stderr.txt but that generates empty files!
To make it clear, I know the script is being executed and that it works, that is not the problem, the problem is getting the output from the script (like from print statements).
If it makes a difference, this started occurring around the time I installed the azure-eventgrid package, through pip install.
Running python in unbuffered mode (with -u) fixes the issue for Azure Batch, but I still cannot understand what change could have caused this. Another similar script that runs in the same VM as the problematic one works just fine without the -u switch.
from Python script called from Azure Batch does not have output
No comments:
Post a Comment