Sunday 30 April 2023

How can I implement a jitter buffer in javascript for realtime audio processing

I have a sender an a receiver running on localhost

both emit and receive on an average interval of 2.89 and 2.92 milliseconds.

the correct constant interval should be 2.90;

So I tried implementing some sort of ringbuffer where buffer size is 128 * N where 128 is the size of the data chunk and N the number of data chunks kept in memory (latency).

I haven't evaluated the time interval between the receival and the processing but according to the data I've managed to calculate a buffer of size N = 2 should be sufficient.

Also my buffer is quite simple it's a FIFO and if a value is received while the buffer is full it replaces the previous chunk.

To get a correct sound I need to make a buffer size of N = 16 so my question is how can I implement a low latency jitter buffer that may be adaptative

I guess currently the buffer adds extra memory for managing an average variation but what I would need is a technique to correct the variation "in place".

my current implementation

_buffer = [];

BUFFER_SIZE = 8;

_isStarted = false;

readIndex = -1

//this callback is triggered any time a packet is received.

_onReceivePacket = ( event ) => {

    let chunks = [];

    //chunk length = 128

    for ( let chunk of event.data ) {

      chunks.push( new Float32Array( chunk ) );

    }

    if ( this._buffer.length < this.BUFFER_SIZE ) {

      this._buffer.unshift( chunks );

      this.readIndex++;

    } else {

      this._buffer.splice( 0, 1 );

      this._buffer.unshift( chunks );

      this._isStarted = true;

    }

}
  //this function copies the buffer into the output stream

  _pullOut ( output ) {

    try {

      for ( let i = 0; i < output.length; i++ ) {

        const channel = output[ i ];

        for ( let j = 0; j < channel.length; j++ ) {

          channel[ j ] = this._buffer[ this.readIndex ][ i ][ j ];

        }

      }

      if ( this.readIndex - 1 !== -1 ) {

        this._buffer.splice( this.readIndex, 1 );

        this.readIndex--;

      }

    } catch ( e ) {

      console.log( e, this._buffer, this.readIndex );

    }

  }


from How can I implement a jitter buffer in javascript for realtime audio processing

No comments:

Post a Comment