Monday, 25 November 2019

Adding UTF-8 support to JS/PHP script

I am working on a page that uses JavaScipt to send data to a PHP script via AJAX POST. The problem is, if the input is in a language that is not Latin based I end-up storing gibberish in the MySQL table. Latin alphabet works fine.

The page itself is capable to rendering UTF-8 characters, if they are in a data provided on page load, it's the post that I struggle with.

اختبار

and save. See the Network POST request in browser's dev tools.

The post is made through the following JS function

function createEmptyStack(stackTitle) {
    return $.ajax({
        type:'POST',
        url:'ajax.php',
        data: {
            "do": 'createEmptyStack',
            newTitle: stackTitle
        },
        dataType: "json"
    });
}

Here's my PHP code.

header('Content-Type: text/html; charset=utf-8');

$newTitle = trim($_POST['newTitle']);

$db->query("
INSERT INTO t1(project_id, label) 
VALUES (".$_SESSION['project_id'].", '".$newTitle."')");

When I check for encoding on the page like this:

mb_detect_encoding($_POST['newTitle'], "auto");

I get result: UTF-8

I also tried the following header:

header("Content-type: application/json; charset=utf-8");

MySQL table collation where the data is supposed to go is set to utf8_general_ci

I have another page that has a form where users populate the same table and it works perfectly fine with ANY language. When I check on the other page why it is capable of inserting similar data into db successfully I see the following above insert query:

mysql_query("SET NAMES utf8");

I've attempted putting the same line above my query that the data still looks gibberish. I also tried the following couple alternatives:

 mysql_query("SET CHARACTER SET utf8 ");

and

mysql_set_charset('utf8', $db);

...but to no avail. I'm stomped. Need help getting it figured out.

Environment:

PHP 5.6.40 (cgi-fcgi)

MySQL 5.6.45


UPDATE

I ran more tests.

I used a phrase "this is a test" in Arabic - هذا اختبار

It seems that ajax.php code works properly. After db insert it returns UTF-8 encoded values, that look like: "\u0647\u0630\u0627 \u0627\u062e\u062a\u0628\u0627\u0631" and the encoding is set to:"UTF-8", however the inserted data in my db table appears as: هذا اختبار

So why am I not jumping to converting my db table to different collation? Couple of reasons: it has nearly .5 mil records and it actually works properly when I go to another page that does very similar INSERT.

Turns out my other page is using ASCII encoding when inserting the data. So it's only natural I try to conver to ASCII on ajax.php. The problem I end-up with blank data. I am so confused now...

Thanks


FIXED: based on a few clues I ended-up rewriting all functions for this page to PDO and it worked!



from Adding UTF-8 support to JS/PHP script

No comments:

Post a Comment