I have raw image data (1000 x 1000 pixels x 3 bytes per pixel) in Python, that I need to send to a HTML page in realtime, at 20 frames per second (this is 57 MB of data per second!).
I already tried the multipart/x-mixed-replace
method (as seen in Sending RGB image data from Python numpy to a browser HTML page), with various encoding: BMP, PNG, JPG. It is quite intensive for the CPU, so I'm trying alternatives.
I am now getting the raw image data directly in JavaScript with binary XHR HTTP requests.
Question: How to (CPU-efficiently) decode binary RGB data from dozens of binary XHR HTTP requests into a <video>
or <img>
or <canvas>
on a HTML page, with Javascript?
oReq = new XMLHttpRequest();
oReq.open("GET", "/imagedata", true);
oReq.responseType = "arraybuffer";
oReq.onload = function (oEvent) {
var arrayBuffer = oReq.response;
if (arrayBuffer) {
var byteArray = new Uint8Array(arrayBuffer);
// update displayed image
}
};
oReq.send(null);
Edit: The method given in @VC.One's comment: var byteArray = new Uint8ClampedArray(arrayBuffer); var imgData = new ImageData(byteArray, 1000, 1000); var ctx = myCanvas.getContext('2d'); ctx.putImageData(imgData, 0, 0);
works and is lighter for the CPU: 5%-6% for the server sending the data vs. 8%-20% with BMP/PNG/JPG encoding. But Chromium now has two processes in parallel for this task, each of them ~ 15% CPU. So the total performance is not much better. Do you see other potential alternatives to efficiently send raw image data from a Python or C++ HTTP server to Chromium?
Also new ImageData(...)
requires a 1000x1000x4 bytes array for R, G, B, A. This requires that I send alpha channel that I don't need; maybe there's a way with ImageData
to only pass a RGB (nxnx3 bytes) array?
Edit 2: the real bottleneck is the XHR HTTP requests between my process #1 and process #2 (Chrome) on the same computer for up to 100 MB/sec. Is there a more direct inter process communication possible between process #1 and Chrome? (some sort of direct memory access?)
See Chrome + another process: interprocess communication faster than HTTP / XHR requests?
from Update frames of a video on a HTML page, from incoming raw image data
No comments:
Post a Comment