Node.js Buffers — Complete Guide to Binary Data Handling

In this tutorial, you will learn about Node.js Buffers. We cover key concepts, practical examples, and best practices to help you master this topic.

Node.js Buffer provides a way to handle raw binary data in memory, essential for file I/O, network protocols, cryptography, and binary data manipulation.

What You'll Learn

By the end of this tutorial, you'll create and manipulate Buffer objects, convert between encodings, slice and copy binary data, and use buffers for practical tasks.

Why Buffers Matter

JavaScript's native typed arrays weren't designed for Node.js I/O. Buffer extends Uint8Array with Node-specific methods for encoding conversion and optimized memory allocation, making it essential for handling binary files and network data.

Real-World Use

An image processing service reads raw image bytes into a Buffer, inspects file headers to validate format, converts between encodings, and writes the processed result to disk.

Buffers Learning Path

flowchart LR
  A[Streams] --> B[Buffers]
  B --> C[Events]
  C --> D[Child Process]
  D --> E[Express.js]
  A --> F{You Are Here}
  style F fill:#f90,color:#fff

Creating Buffers

import { Buffer } from "node:buffer";

// From string
const buf1 = Buffer.from("Hello Node.js", "utf8");
console.log(buf1);  // <Buffer 48 65 6c 6c 6f 20 4e 6f 64 65 2e 6a 73>

// Allocate (zero-filled)
const buf2 = Buffer.alloc(10);  // 10 zero-filled bytes
console.log(buf2);  // <Buffer 00 00 00 00 00 00 00 00 00 00>

// Allocate (uninitialized - faster but may contain old data)
const buf3 = Buffer.allocUnsafe(10);
console.log(buf3);  // <Buffer ... old data>

Always use Buffer.alloc for new buffers. Buffer.allocUnsafe is faster but may expose sensitive data if not overwritten.

Reading and Writing Buffers

const buf = Buffer.alloc(8);
buf.writeUInt16BE(0x1234, 0);  // Write 16-bit value at byte 0
buf.writeUInt32LE(0xDEADBEEF, 2);  // Write 32-bit value at byte 2

console.log(buf);  // <Buffer 12 34 ef be ad de 00 00>
console.log(buf.readUInt16BE(0));  // 4660 (0x1234)
console.log(buf.readUInt32LE(2));  // 3735928559 (0xDEADBEEF)

Encoding Conversion

const buf = Buffer.from("Hello", "utf8");
console.log(buf.toString("hex"));         // 48656c6c6f
console.log(buf.toString("base64"));      // SGVsbG8=
console.log(buf.toString("ascii"));       // Hello
console.log(buf.toString("utf8"));        // Hello

Buffer Slicing

Slice creates a new view (not a copy) into the original buffer.

const original = Buffer.from("Hello World");
const slice = original.subarray(0, 5);
console.log(slice.toString());  // Hello

// Modifying the slice affects the original
slice[0] = 104;  // 'h'
console.log(original.toString());  // hello World

Buffer Comparison and Concatenation

const buf1 = Buffer.from("ABC");
const buf2 = Buffer.from("ABC");
const buf3 = Buffer.from("ABD");

console.log(buf1.equals(buf2));  // true
console.log(buf1.equals(buf3));  // false

console.log(Buffer.compare(buf1, buf3));  // -1 (buf1 comes first)
const combined = Buffer.concat([buf1, buf3]);
console.log(combined.toString());  // ABCABD

Buffer Length vs String Length

const str = "Hello";
const emoji = "🎉";
console.log(str.length);         // 5
console.log(Buffer.from(str).length);  // 5
console.log(emoji.length);       // 2 (JS string length)
console.log(Buffer.from(emoji).length);  // 4 (UTF-8 byte length)

UTF-8 characters like emoji take 4 bytes. Always use Buffer.from(str).length for accurate byte counts.

Common Mistakes

1. Using new Buffer() (Deprecated)

new Buffer(10) may allocate uninitialized memory containing sensitive data. Use Buffer.alloc() or Buffer.from().

2. Confusing String Length and Buffer Length

String .length counts UTF-16 code units. Buffer .length counts bytes. They differ for multi-byte characters.

3. Assuming toString Without Encoding

buf.toString() defaults to utf8. Binary data may produce garbled output. Always specify the encoding.

4. Modifying Shared Buffer Slices

subarray shares memory with the parent. Copy with Buffer.from(slice) if you need independent data.

5. Off-by-One Errors in read/write Position

Buffer write operations at a wrong offset corrupt data. Validate offsets against buffer length.

Practice Questions

1. What is the difference between Buffer.alloc and Buffer.allocUnsafe?

Buffer.alloc fills memory with zeros (safe). Buffer.allocUnsafe is faster but may contain old data (unsafe for sensitive information).

2. How do you convert a Buffer to a base64 string?

buf.toString("base64")

3. Does Buffer.subarray create a new copy?

No. It creates a new view that shares the same underlying memory. Modifications affect the original buffer.

4. What is the byte length of the string "cafe"?

4 bytes in UTF-8 (all ASCII characters are 1 byte each).

5. Challenge: Write a function that reads a file header (first 4 bytes) and returns whether it's a PNG file (starts with 0x89504E47).

import fs from "node:fs";
import { Buffer } from "node:buffer";
function isPNG(filePath) {
  const fd = fs.openSync(filePath, "r");
  const header = Buffer.alloc(4);
  fs.readSync(fd, header, 0, 4, 0);
  fs.closeSync(fd);
  return header.readUInt32BE(0) === 0x89504E47;
}
console.log(isPNG("image.png"));  // true or false

FAQ

Mini Project: File Type Detector

Create a utility that reads the magic bytes of a file and detects its type.

import fs from "node:fs";
import { Buffer } from "node:buffer";
const MAGIC_NUMBERS = {
  "89504E47": "PNG",
  "FFD8FF": "JPEG",
  "25504446": "PDF",
  "504B0304": "ZIP/DOCX"
};
function detectFileType(filePath) {
  const fd = fs.openSync(filePath, "r");
  const header = Buffer.alloc(4);
  fs.readSync(fd, header, 0, 4, 0);
  fs.closeSync(fd);
  const hex = header.toString("hex").toUpperCase();
  for (const [magic, type] of Object.entries(MAGIC_NUMBERS)) {
    if (hex.startsWith(magic)) return type;
  }
  return "Unknown";
}
console.log(detectFileType("document.pdf"));  // PDF

What's Next

Node.js Events Node.js Error Handling Express.js