Wednesday 23 September 2020

The length of a compressed Java String is not equal to the content-length when it is sent as a WebSocket message

I am trying to reduce bandwidth consumption by compressing the JSON String I am sending through the WebSocket from my Springboot application to the browser client (this is on top of permessage-deflate WebSocket extension). This scenario uses the following JSON String which has a length of 383 characters:

{"headers":{},"body":{"message":{"errors":{"password":"Password length must be at least 8 characters.","retype":"Retype Password cannot be null.","username":"Username length must be between 6 to 64 characters."},"links":[],"success":false,"target":{"password":"","retype":"","username":""}},"target":"/user/session/signup"},"statusCode":"UNPROCESSABLE_ENTITY","statusCodeValue":422}

To benchmark, I send both compressed and uncompressed String from the server like so:

Object response = …,

SimpMessageHeaderAccessor simpHeaderAccessor =
    SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE);
simpHeaderAccessor.setSessionId(sessionId);
simpHeaderAccessor.setContentType(new MimeType("application", "json",
    StandardCharsets.UTF_8));
simpHeaderAccessor.setLeaveMutable(true);
// Sends the uncompressed message.
messagingTemplate.convertAndSendToUser(sessionId, uri, response,
    simpHeaderAccessor.getMessageHeaders());

ObjectMapper mapper = new ObjectMapper();
String jsonString;

try {
    jsonString = mapper.writeValueAsString(response);
}
catch(JsonProcessingException e) {
    jsonString = response.toString();
}

log.info("The payload is application/json.");
log.info("uncompressed payload (" + jsonString.length() + " character):");
log.info(jsonString);

String lzStringCompressed = LZString.compress(jsonString);
simpHeaderAccessor = SimpMessageHeaderAccessor.create(SimpMessageType.MESSAGE);
simpHeaderAccessor.setSessionId(sessionId);
simpHeaderAccessor.setContentType(new MimeType("text", "plain",
    StandardCharsets.UTF_8));
simpHeaderAccessor.setLeaveMutable(true);
// Sends the compressed message.
messagingTemplate.convertAndSendToUser(sessionId, uri, lzStringCompressed,
    simpHeaderAccessor.getMessageHeaders());

log.info("The payload is text/plain.");
log.info("compressed payload (" + lzStringCompressed.length() + " character):");
log.info(lzStringCompressed);

Which logs the following lines in the Java console:

The payload is application/json.
uncompressed payload (383 character):
{"headers":{},"body":{"message":{"errors":{"password":"Password length must be at least 8 characters.","retype":"Retype Password cannot be null.","username":"Username length must be between 6 to 64 characters."},"links":[],"success":false,"target":{"password":"","retype":"","username":""}},"target":"/user/session/signup"},"statusCode":"UNPROCESSABLE_ENTITY","statusCodeValue":422}
The payload is text/plain.
compressed payload (157 character):
??????????¼??????????????p??!-??7??????????????????????????????????u??????????????????????·}???????????????????????????????????????/?┬R??b,??????m??????????

Then browser receives the two messages sent by the server and captured by this javascript:

stompClient.connect({}, function(frame) {
    stompClient.subscribe(stompClientUri, function(payload) {
        try {
            JSON.parse(payload.body);
            console.log("The payload is application/json.");
            console.log("uncompressed payload (" + payload.body.length + " character):");
            console.log(payload.body);

            payload = JSON.parse(payload.body);
        } catch (e) {
            try {
                payload = payload.body;
                console.log("The payload is text/plain.");
                console.log("compressed payload (" + payload.length + " character):");
                console.log(payload);

                var decompressPayload = LZString.decompress(payload);
                console.log("decompressed payload (" + decompressPayload.length + " character):");
                console.log(decompressPayload);

                payload = JSON.parse(decompressPayload);
            } catch (e) {
            } finally {
            }
        } finally {
        }
    });
});

Which displays the following lines in the browser's debug console:

The payload is application/json.
uncompressed payload (383 character):
{"headers":{},"body":{"message":{"errors":{"password":"Password length must be at least 8 characters.","retype":"Retype Password cannot be null.","username":"Username length must be between 6 to 64 characters."},"links":[],"success":false,"target":{"password":"","retype":"","username":""}},"target":"/user/session/sign-up"},"statusCode":"UNPROCESSABLE_ENTITY","statusCodeValue":422}
The payload is text/plain.
compressed payload (157 character):
ᯡࠥ䅬ࢀጨᎡ乀ஸ̘͢¬ߑ䁇啰˸⑱ᐣ䱁ሢ礒⽠݉ᐮ皆⩀p瑭漦!-䈠ᷕ7ᡑ刡⺨狤灣મ啃嵠ܸ䂃ᡈ硱䜄ቀρۯĮニᴴဠ䫯⻖֑点⇅劘畭ᣔ奢⅏㛥⡃Ⓛ撜u≂㥋╋ၲ⫋䋕᪒丨ಸ䀭䙇Ꮴ吠塬昶⬻㶶Т㚰ͻၰú}㙂᥸沁⠈ƹ⁄᧸㦓ⴼ䶨≋愐㢡ᱼ溜涤簲╋㺮橿䃍砡瑧ᮬ敇⼺ℙ滆䠢榵ⱀ盕ີ‣Ш眨રą籯/ሤÂR儰Ȩb,帰Ћ愰䀥․䰂m㛠ளǀ䀭❖⧼㪠Ө柀䀠 
decompressed payload (383 character):
{"headers":{},"body":{"message":{"errors":{"password":"Password length must be at least 8 characters.","retype":"Retype Password cannot be null.","username":"Username length must be between 6 to 64 characters."},"links":[],"success":false,"target":{"password":"","retype":"","username":""}},"target":"/user/session/sign-up"},"statusCode":"UNPROCESSABLE_ENTITY","statusCodeValue":422}

At this point I can now verify that whatever String value my Springboot application compresses, the browser can able to decompress and get the original String. There is a problem though. When I inspected the browser debugger if the size of the transferred message was actually reduced, it tells me that isn't.

Here is the raw uncompressed message (598B):

a["MESSAGE destination:/user/session/broadcast
content-type:application/json;charset=UTF-8
subscription:sub-0
message-id:5lrv4kl1-1
content-length:383

{"headers":{},"body":{"message":{"errors":{"password":"Password length must be at least 8 characters.","retype":"Retype Password cannot be null.","username":"Username length must be between 6 to 64 characters."},"links":[],"success":false,"target":{"password":"","retype":"","username":""}},"target":"/user/session/sign-up"},"statusCode":"UNPROCESSABLE_ENTITY","statusCodeValue":422}

While this is the raw compressed message (589B):

a["MESSAGE destination:/user/session/broadcast
content-type:text/plain;charset=UTF-8
subscription:sub-0
message-id:5lrv4kl1-2
content-length:425

á¯¡à ¥ä¬à¢á¨á¡ä¹à®¸Ì͢¬ßäå°Ë¸â±á£ä±á¢ç¤â½Ýá®çâ©pç­æ¼¦!-ä á·7á¡å¡âº¨ç¤ç£àª®ååµÜ¸äá¡ç¡±äáÏۯĮãá´´á䫯â»Öç¹âåç­á£å¥¢âã¥â¡âæuâã¥âá²â«äáªä¸¨à²¸ä­äá¤å塬æ¶â¬»ã¶¶Ð¢\u2029ã°Í»á°Ãº}ã᥸æ²âƹâ᧸ã¦â´¼ä¶¨âæ㢡ᱼæºæ¶¤ç°²â㺮橿äç¡ç§á®¬æ⼺âæ»ä¢æ¦µâ±çີâ£Ð¨ç¨àª°Ä籯/á¤ÃRå°È¨b,帰Ðæ°ä¥â¤ä°mãளÇä­â⧼㪠Өæä  \u0000"]

The debug console indicates that the uncompressed message was transferred with the size of 598B, with 383 character as the message payload's size (indicated by the content-length header). While on the other hand, the compressed message was transferred with a total size of 589B, 9B smaller than the uncompressed one, with 425 character as the message payload's size. I have several questions:

  1. Is the content-length of the STOMP message indicated in bytes, or in characters?
  2. Why does the content-length of the uncompressed message, which is 383, smaller than that of the compressed message, which is 425?
  3. Does this mean reducing the character length does not always necessarily means reducing the size?
  4. Why does the content-length of the compressed message, which is 425, not the same with the value returned in the Java console (using lzStringCompressed.length()) which is 157, considering that the uncompressed message was transferred with a content-length of 383, which is the same length in Java console. Both too are transferred with charset=UTF-8 encoding.
  5. Why does the content-length of the compressed message, which is 425, not the same with value returned in the Java console (using lzStringCompressed.length()) which is 157 but the JavaScript code payload.length returns 157, not 425?
  6. If it really gets bloated during the transfer, why does the message with application/json remained unaffected and only the plain/text gets bloated?

While the 9B difference is still a difference, I am reconsidering if the overhead cost for compressing/decompressing the message is worth to keep. I have to test other String values for that.



from The length of a compressed Java String is not equal to the content-length when it is sent as a WebSocket message

No comments:

Post a Comment