DEV Community

Cover image for Demystifying Buffers in Node.js
angDecoder
angDecoder

Posted on

Demystifying Buffers in Node.js

Table of Contents

1. What is Buffer?
2. What is Character Encoding?
3. Buffer in Node.js



In this blog series, we'll unravel the mystery surrounding buffers, exploring their crucial role in optimising everything from streaming to web browsing.

1. What is Buffer?

Buffers --> These are temporary storage area in memory used to hold data while it's being transferred from one place to another.
In Node.js, Buffer objects are used to represent a fixed-length sequnce of bytes.

Let us say, we want to copy a file of size 10GB, from one device to another device. The device speed may vary, which would lead to various problems like :

  • Memory Overload --> Reading the entire file into memory at once could easily overwhelm the available memory resource.

  • Performance Degradation --> As the system would spend much of it's time in I/O operation.

  • Data corruption --> If error occurs during file transfer, there would be no way to maintain data integrity and some data might corrupted.


2. What is Character Encoding?

Character Encoding --> It is the process of representing characters from a written language, numbers, and symbols as binary data (bits and bytes) for storage or transmission in a digital system.

It is used to represent a sequence of bits into a character. For eg :-
'11001001' might represent character 'a' in some encoding while 'd' in another encoding.

Here are some basic definition :-

  • Character Set --> A character set is a collection of characters, such as letters, digits, punctuation marks, and special symbols, used in written communication.

  • Binary Representation --> Since computer can only understand 0's and 1's, each character in character set in give a unique binary code.

  • Code Pages And Standards --> Different character encoding use different encoding schemes for encoding. For eg -> ASCII uses 7-bits to represent 128 characters and UTF-8 uses 8-bits to represent characters.

Some examples of encoding are :- UTF-8, UTF-16, ASCII etc. In Node.js default encoding is UTF-8.


3. Buffer in Node.js

Buffer objects are used to represent a fixed-length sequence of bytes. The Buffer class is a subclass of JavaScript's Uint8Array class and extends it with methods that cover additional use cases.


a. Buffer.alloc(size,fill,encoding) --> It creates a new buffer with given size.

  • size : The desired length of the new Buffer.
  • fill : A value to pre-fill the new Buffer with. Default: 0.
  • encoding : Encoding to be used. Default 'UTF-8'.

const { Buffer } = require('node:buffer');
// Buffer is available globally but it is 
// recommended to import explicitly


const buff1 = Buffer.alloc(5);
console.log(buff1);
// Buffer allocated 5 size filled with zero
// output is represented as hexadecimal number
// OUTPUT : <Buffer 00 00 00 00 00>


const buff2 = Buffer.alloc(5,1);
console.log(buff2);
// buffer allocated 5 size filled with 1
// OUTPUT : <Buffer 01 01 01 01 01>

Enter fullscreen mode Exit fullscreen mode


b. Buffer.from(array or string) --> It takes array or string as an argument and creates the buffer size and value accordingly.


const { Buffer } = require('node:buffer');

const buff3 = Buffer.from('hello','utf-8');
console.log(buff3);
// buffer size allocated for string 'hello'
// with character encoding 'utf-8'(default)
// OUTPUT : <Buffer 68 65 6c 6c 6f>

const buff4 = Buffer.from([0x7f,17,0]);
console.log(buff4);
// buffer size and value according to array
// array elements can be number(integer OR hexadecimal)
// OUTPUT : <Buffer 7f 11 00>

Enter fullscreen mode Exit fullscreen mode


c. Buffer.allocUnsafe(size) --> It creates a buffer of given size but with random values.( NOT RECOMMENDED ). It is little faster than "Buffer.alloc()" method


const { Buffer } = require('node:buffer');

const buff5 = Buffer.allocUnsafe(5);
console.log(buff5);
// faster than '.alloc()' method
// OUTPUT : <Buffer 60 67 6c 6d 6e>

Enter fullscreen mode Exit fullscreen mode


d. Buffer.compare(buff1,buff2) --> Mainly used when we want to sort array of Buffers. It's output could vary :-

  • 0 is returned if buff2 is the same as buff1
  • 1 is returned if buff2 should come before buff1 when sorted.
  • -1 is returned if buff2 should come after buff1 when sorted.

const { Buffer } = require('buffer');

const buff6 = Buffer.from('0123');
const buff7 = Buffer.from('1234');

console.log(Buffer.compare(buff6,buff7));
// OUTPUT : -1

Enter fullscreen mode Exit fullscreen mode


e. Buffer.toString(encoding) --> This method converts a given buffer to string with given encoding. DEFAULT ENCODING : UTF-8


const { Buffer } = require('buffer');

const buff8 = Buffer.from('hello');
console.log(buff8.toString());
// OUTPUT : hello

Enter fullscreen mode Exit fullscreen mode

Top comments (0)