Thursday, 12 July 2018

Nodejs worker threads shared object/store

So, I was reading some stuff regarding nodejs and I was amazed when I came across Worker Threads.

Having threads in my opinion is a great plus especially if you combine it with shared memory access. As you might think already -> SharedArrayBuffer...

Yeap that's what I thought. So The first thing that came into my mind was to give it a little test and try to implement a simple store (a simple object for now) that would be shared among the threads.

The question is, (unless I'm missing something here) how can you make an object accessible from n threads with the use of SharedArrayBuffer?

I know that for a simple Uint32Array is doable, but regarding the object what can be done?

At first I thought to convert it to a Uint32Array as you may see bellow, but even looking at the damn source code makes me wanna cry...

const {
    Worker,
    isMainThread,
    workerData
} = require('worker_threads');

const store = {
    ks109: {
        title: 'some title 1',
        description: 'some desciption 1',
        keywords: ['one', 'two']
    },
    ks110: {
        title: 'some title 2',
        description: 'some desciption 2',
        keywords: ['three', 'four']
    },
    ks111: {
        title: 'some title 3',
        description: 'some desciption 3',
        keywords: ['five', 'six']
    }
}

const shareObject = (obj) => {

    let OString = JSON.stringify(obj);
    let SABuffer = new SharedArrayBuffer(Int32Array.BYTES_PER_ELEMENT * OString.length);
    let sArray = new Int32Array(SABuffer);

    for (let i = 0; i < OString.length; i++) {
        sArray[i] = OString.charCodeAt(i);
    }

    return sArray;

}

if (isMainThread) {

    const sharedStore = shareObject(store);

    console.log('[Main][Data Before]:', sharedStore.slice(-10));

    let w = new Worker(__filename, {
        workerData: sharedStore
    });

    w.on('message', (msg) => {
        console.log(`[Main][Thread message]: ${msg}`);
    });
    w.on('error', (err) => {
        console.error(`[Main][Thread error] ${err.message}`);
    });
    w.on('exit', (code) => {
        if (code !== 0) {
            console.error(`[Main][Thread unexpected exit]: ${code}`);
        }
    });

    setInterval(() => {
        // At some point you ll see a value change,
        // it's 'six' becoming 'sax' (big time!) 
        console.log('[Main][Data Check]:', sharedStore.slice(-10));
    }, 1000);

} else {

    let str = String.fromCharCode.apply(this, workerData);
    let obj = JSON.parse(str);

    obj.ks111.keywords[1] = 'sax'; // big time!

    let OString = JSON.stringify(obj);

    for (let i = 0; i < OString.length; i++) {
        workerData[i] = OString.charCodeAt(i);
    }

}

In conclusion, shared object among threads in nodejs 10.5.0, is it possible? how?



from Nodejs worker threads shared object/store

No comments:

Post a Comment