I'm building a mobile app using Flutter that translates American Sign Language (ASL) to written English. As a prototype, I've limited my scope to translating photos of single alphanumeric characters. I have two parts to the project right now:
-
A python script that takes the path to an image, processes it with Torch/Torchvision and a custom model, and returns the alphanumeric character that is most likely to have been in the image. This part works great.
-
A Flutter front end that connects to the user's camera and saves the path to a taken image.
Q: What is the best way to connect these two parts of my project?
So far, I've tried two methods:
- Using Starflut, which allows me to call Python code from Flutter and receive return values. I've tested it with some basic Python code (no dependencies) and it works great. However, for my more complex use case, where my Python code imports "torch", the code crashes.
Error Message:
:/data/data/com.example.asl/files/flutter_assets/starfiles/image_prediction.py, run failed
ModuleNotFoundError: No module named 'torch'
This happens when calling the following line of code:
var result = await widget.srvGroup.loadRawModule("python", "",
resPath + "/flutter_assets/starfiles/" + "image_prediction.py", false);
- I've also tried Pytorch_mobile Flutter package, but beyond not being working exactly in the way I want and implemented in my own Python script, also crashes.
Error Message:
java.lang.NullPointerException: Attempt to invoke virtual method 'org.pytorch.IValue org.pytorch.Module.forward(org.pytorch.IValue[])' on a null object reference
This happens when calling the following line of code:
String prediction =
await TorchMobile.getPrediction(_image, maxWidth: 256, maxHeight: 256);
Any suggestions as to how to get Starflut functioning would be ideal, as I'd like to use my Python interface, but if that's not possible, I'd love suggestions as to how to approach this problem.
from Flutter- Connecting to Python/Pytorch Backend
No comments:
Post a Comment