Node.js File System Deep Dive — Complete Guide to fs and fs/promises

In this tutorial, you will learn about Node.js File System Deep Dive. We cover key concepts, practical examples, and best practices to help you master this topic.

Node.js file system deep dive covers the fs module and its promise-based fs/promises counterpart, including file descriptors, streams, directory walking, file watching, and permission handling.

What You'll Learn

By the end of this tutorial, you'll use fs and fs/promises for advanced file operations, work with file descriptors, watch files for changes, walk directories recursively, and manage file permissions.

Why Deep File System Matters

File operations are the most common I/O in Node.js. Understanding fs internals helps you avoid memory issues, handle concurrent access, and build reliable file-processing pipelines.

Real-World Use

A log aggregator watches multiple log files with fs.watch, processes new lines in real time, rotates files when they exceed size limits, and archives to cloud storage.

File System Deep Path

flowchart LR
  A[Buffers] --> B[File System Deep]
  B --> C[Streams]
  B --> D[Path Module]
  C --> E[Error Handling]
  B --> F{You Are Here}
  style F fill:#f90,color:#fff

Working with File Descriptors

Open files with fs.open for fine-grained control over read/write operations.

const fs = require("node:fs");
fs.open("/tmp/data.txt", "r", (err, fd) => {
  if (err) throw err;
  const buffer = Buffer.alloc(1024);
  fs.read(fd, buffer, 0, 1024, 0, (err, bytesRead) => {
    if (err) throw err;
    console.log("Read", bytesRead, "bytes:", buffer.toString("utf8", 0, bytesRead));
    fs.close(fd, (err) => {
      if (err) throw err;
    });
  });
});

fs/promises API

Use the promise-based API for cleaner async file operations.

const fs = require("node:fs/promises");
async function manageFile(path) {
  const handle = await fs.open(path, "r+");
  const stats = await handle.stat();
  console.log("File size:", stats.size);
  const buffer = Buffer.alloc(stats.size);
  await handle.read(buffer, 0, stats.size, 0);
  console.log("Content:", buffer.toString("utf8"));
  await handle.close();
}
await manageFile("/tmp/data.txt");

Directory Operations

Walk directories recursively, create nested directories, and check paths.

const fs = require("node:fs/promises");
const path = require("node:path");
async function walkDir(dirPath) {
  const entries = await fs.readdir(dirPath, { withFileTypes: true });
  const results = [];
  for (const entry of entries) {
    const fullPath = path.join(dirPath, entry.name);
    if (entry.isDirectory()) {
      results.push(...await walkDir(fullPath));
    } else {
      results.push(fullPath);
    }
  }
  return results;
}
const files = await walkDir("/var/log");
console.log("Log files:", files.length);

File Watching

Monitor files and directories for changes using fs.watch.

const fs = require("node:fs");
const watcher = fs.watch("/var/log/app.log", (eventType, filename) => {
  console.log(`File ${filename} changed: ${eventType}`);
});
fs.watch("/var/log", { recursive: true }, (eventType, filename) => {
  console.log(`Directory event: ${eventType} on ${filename}`);
});
// Cleanup
watcher.close();

File Permissions and Ownership

Check and modify file permissions using chmod and chown.

const fs = require("node:fs/promises");
async function setSecurePermissions(filePath) {
  const stats = await fs.stat(filePath);
  console.log("Current mode:", stats.mode.toString(8));
  await fs.chmod(filePath, 0o600);
  console.log("Changed to user read/write only (600)");
}
await setSecurePermissions("/tmp/sensitive.txt");

Common Mistakes

1. Not Handling ENOENT Errors

File not found errors must be caught specifically. Check file existence before operations.

2. Forgetting to Close File Descriptors

Unclosed descriptors leak system resources. Use fs.open with try-finally or the promise-based API.

3. Using readFile for Large Files

readFile loads the entire file into memory. Use createReadStream for files over 50MB.

4. Ignoring File Locking

Concurrent writes to the same file cause corruption. Use flock or append-only patterns.

5. Assuming fs.watch Is Cross-Platform

fs.watch behavior varies between Linux (inotify) and macOS (FSEvents). Test on all platforms.

Practice Questions

1. What is the difference between fs and fs/promises?

fs uses callbacks. fs/promises provides Promise-based API for use with async-await.

2. How do you check if a path is a file or directory?

Use fs.stat() and check the isFile() and isDirectory() methods on the result.

3. What is the recommended way to read large files?

Use fs.createReadStream() to Process data in chunks instead of loading the entire file into memory.

4. How does fs.watch differ from fs.watchFile?

fs.watch uses OS-native file system events (inotify, FSEvents). fs.watchFile polls for changes periodically.

5. Challenge: Build a utility that watches a directory and logs all file changes.

const fs = require("node:fs");
const path = require("node:path");
function watchDirectory(dir) {
  const watchers = new Map();
  fs.readdirSync(dir).forEach((file) => {
    const filePath = path.join(dir, file);
    const watcher = fs.watch(filePath, (event) => {
      console.log(`[${file}] ${event}`);
    });
    watchers.set(filePath, watcher);
  });
  return () => watchers.forEach((w) => w.close());
}
const cleanup = watchDirectory("/tmp/logs");
setTimeout(cleanup, 60000);

FAQ

Mini Project: File Change Tracker

Build a utility that monitors file changes and saves diffs.

const fs = require("node:fs");
const path = require("node:path");
class FileTracker {
  constructor(filePath) {
    this.filePath = filePath;
    this.content = fs.readFileSync(filePath, "utf8");
  }
  start() {
    fs.watch(this.filePath, (event) => {
      if (event === "change") {
        const newContent = fs.readFileSync(this.filePath, "utf8");
        if (newContent !== this.content) {
          console.log(`[${new Date().toISOString()}] ${this.filePath} changed`);
          this.content = newContent;
        }
      }
    });
  }
}

What's Next

Node.js Path Module Node.js Streams Node.js Buffers