I'm writing an application that renders a sequence of pictures received in real-time from a remote TCP connection into an ImageView
element. The stream is composed of single frames encoded in PGM format and sent at 9Hz I tought that a very low frame rate like this should be handled easily using a background Service
that sends fully decoded bitmap to my MainActivity
.
Here's my VideoService
(I'm posting just run()
method since I think it's the only one of some interest):
public void run() {
InetAddress serverAddr = null;
try {
serverAddr = InetAddress.getByName(VIDEO_SERVER_ADDR);
} catch (UnknownHostException e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
return;
}
Socket socket = null;
BufferedReader reader = null;
do {
try {
socket = new Socket(serverAddr, VIDEO_SERVER_PORT);
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
boolean frameStart = false;
LinkedList<String> frameList = new LinkedList<>();
while (keepRunning) {
final String message = reader.readLine();
if (!frameStart && message.startsWith("F"))
frameStart = true;
else if (frameStart && message.startsWith("EF")) {
frameStart = false;
final Bitmap bitmap = Bitmap.createBitmap(IR_FRAME_WIDTH, IR_FRAME_HEIGHT, Bitmap.Config.ARGB_8888);
final Canvas canvas = new Canvas(bitmap);
final String[] data = frameList.toArray(new String[frameList.size()]);
canvas.drawBitmap(bitmap, 0, 0, null);
//Log.d(this.getClass().getName(), "IR FRAME COLLECTED");
if ((data.length - 6) == IR_FRAME_HEIGHT) {
float grayScaleRatio = Float.parseFloat(data[2].trim()) / 255.0f;
for (int y = 0; y < IR_FRAME_HEIGHT; y++) {
final String line = data[y + 3];
final String[] points = line.split("\\s+");
if (points.length == IR_FRAME_WIDTH) {
for (int x = 0; x < IR_FRAME_WIDTH; x++) {
final float grayLevel = Float.parseFloat(points[x]) / grayScaleRatio;
Paint paint = new Paint();
paint.setStyle(Paint.Style.FILL);
final int level = (int)grayLevel;
paint.setColor(Color.rgb(level, level, level));
canvas.drawPoint(x, y, paint);
}
} else
Log.d(this.getClass().getName(), "Malformed line");
}
final Intent messageIntent = new Intent();
messageIntent.setAction(VIDEO_BROADCAST_KEY);
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream);
bitmap.recycle();
messageIntent.putExtra(VIDEO_MESSAGE_KEY, stream.toByteArray());
stream.close();
sendBroadcast(messageIntent);
} else
Log.d(this.getClass().getName(), "Malformed data");
frameList.clear();
} else if (frameStart)
frameList.add(message);
}
Thread.sleep(VIDEO_SERVER_RESPAWN);
} catch (Throwable e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
}
} while (keepRunning);
if (socket != null) {
try {
socket.close();
} catch (Throwable e) {
Log.e(getClass().getName(), e.getMessage());
e.printStackTrace();
}
}
}
The message
is a line coming from the following text:
F
P2
160 120
1226
193 141 158 152 193 186 171 177 186 160 195 182 ... (160 times)
.
. (120 lines)
.
278 248 253 261 257 284 310 304 304 272 227 208 ... (160 times)
EF
In MainActivity I handle this trough this code:
class VideoReceiver extends BroadcastReceiver {
final public Queue<Bitmap> imagesQueue = new LinkedList<>();
@Override
public void onReceive(Context context, Intent intent) {
try {
//Log.d(getClass().getName(), "onReceive() called");
final byte[] data = intent.getByteArrayExtra(VideoService.VIDEO_MESSAGE_KEY);
final Bitmap bitmap = BitmapFactory.decodeByteArray(data,0,data.length);
imagesQueue.add(bitmap);
runOnUiThread(updateVideoTask);
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
updateVideoTask
task is defined like this:
updateVideoTask = new Runnable() {
public void run() {
if (videoReceiver == null) return;
if (!videoReceiver.imagesQueue.isEmpty())
{
final Bitmap image = videoReceiver.imagesQueue.poll();
if (image == null) return;
videoView.setImageBitmap(image);
Log.d(this.getClass().getName(), "Images to spool: " + videoReceiver.imagesQueue.size());
}
}
};
Unluckly when I run the application I notice a very low frame rate and a very big delay. I cannot argue what's going on. The only hints I got from logcat are these lines:
2019-05-20 16:37:08.817 29566-29580/it.tux.gcs I/art: Background sticky concurrent mark sweep GC freed 88152(3MB) AllocSpace objects, 3(52KB) LOS objects, 22% free, 7MB/10MB, paused 3.937ms total 111.782ms
2019-05-20 16:37:08.832 29566-29587/it.tux.gcs D/skia: Encode PNG Singlethread : 13003 us, width=160, height=120
even with the sum of all this delay (140 ms) the app should sustain a frame rate of more than 5Hz while am getting 0.25Hz or even worse.
After some investigation I found that moving:
Paint paint = new Paint();
paint.setStyle(Paint.Style.FILL);
out of the nested loops prevent GC from being invoked so frequently and I found another major source of delay in this line:
final String[] points = line.split("\\s+");
it burns out 2ms per time so I decided to go for something less smart but faster:
final String[] points = line.split(" ");
Anyway it's still not enough.. the code between:
canvas.drawBitmap(bitmap, 0, 0, null);
and
sendBroadcast(messageIntent);
still consume more than 200ms ... how can I do better than this?
from Optimization in bitmap creation
No comments:
Post a Comment