Node.js Child Process Module
What is the Child Process Module?
The Child Process module is a built-in Node.js module that allows you to create and manage child processes.
It provides several ways to execute external commands and communicate with subprocess instances.
This capability is essential for tasks like:
- Running system commands from your Node.js application
- Executing CPU-intensive tasks in separate processes
- Running multiple processes in parallel to utilize multiple CPU cores
- Interfacing with external programs and scripts
Importing the Child Process Module
The Child Process module is included in Node.js by default.
You can use it by requiring it in your script:
const childProcess = require('child_process');
// Or using destructuring to access specific methods
const { exec, spawn, fork } = require('child_process');
Methods for Creating Child Processes
The Child Process module provides four primary methods for creating and managing child processes, each with different behaviors and use cases:
| Method | Description | Use Case |
|---|---|---|
exec() |
Spawns a shell and executes a command, buffering the output | When you need to run a shell command and get the entire output at once |
execFile() |
Similar to exec() but doesn't spawn a shell |
More efficient for executing file-based commands without shell interpretation |
spawn() |
Spawns a new process without creating a shell, with streaming I/O | When dealing with long-running processes or large output |
fork() |
A special case of spawn() for creating Node.js processes |
When you need to run another Node.js module as a separate process with IPC |
The exec() Method
The exec() method creates a shell and executes a command within that shell.
It buffers the entire output and provides it via a callback when the command completes.
const { exec } = require('child_process');
// Execute the 'ls -la' command (or 'dir' on Windows)
const command = process.platform === 'win32' ? 'dir' : 'ls -la';
exec(command, (error, stdout, stderr) => {
if (error) {
console.error(`Error executing command: ${error.message}`);
return;
}
if (stderr) {
console.error(`Command stderr: ${stderr}`);
}
console.log(`Command output:\n${stdout}`);
});
// With options
exec('echo $HOME', {
env: { HOME: '/custom/home/directory' }
}, (error, stdout, stderr) => {
console.log(`Custom home directory: ${stdout.trim()}`);
});
Warning: Never pass unsanitized user input to exec() as it runs commands with full shell syntax, which can lead to command injection attacks.
exec() with Promise
Using a promise wrapper to handle the callback:
const { exec } = require('child_process');
const util = require('util');
// Convert exec to a promise-based function
const execPromise = util.promisify(exec);
async function executeCommand(command) {
try {
const { stdout, stderr } = await execPromise(command);
if (stderr) {
console.error(`Command stderr: ${stderr}`);
}
console.log(`Command output:\n${stdout}`);
return stdout;
} catch (error) {
console.error(`Error executing command: ${error.message}`);
throw error;
}
}
// Using the promise-based function
executeCommand('node --version')
.then(version => console.log(`Node.js version: ${version.trim()}`))
.catch(err => console.error('Failed to get Node.js version'));
The execFile() Method
The execFile() method is similar to exec(), but it doesn't spawn a shell.
It's more efficient for executing external binaries.
const { execFile } = require('child_process');
// Execute 'node' with arguments
execFile('node', ['--version'], (error, stdout, stderr) => {
if (error) {
console.error(`Error executing file: ${error.message}`);
return;
}
console.log(`Node.js version: ${stdout.trim()}`);
});
// On Windows, execute a batch file
if (process.platform === 'win32') {
execFile('C:\\Windows\\System32\\cmd.exe', ['/c', 'echo Hello from batch!'], (error, stdout, stderr) => {
if (error) {
console.error(`Error: ${error.message}`);
return;
}
console.log(`Output: ${stdout.trim()}`);
});
}
Note: execFile() is more secure than exec() for running commands with user input, as it doesn't process shell metacharacters.
The spawn() Method
The spawn() method launches a new process with the given command.
Unlike exec(), it doesn't buffer the output, instead providing stream-based access to stdout and stderr.
const { spawn } = require('child_process');
// Spawn a process to list files
const ls = process.platform === 'win32'
? spawn('cmd', ['/c', 'dir'])
: spawn('ls', ['-la']);
// Handle process output streams
ls.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
ls.stderr.on('data', (data) => {
console.error(`stderr: ${data}`);
});
ls.on('close', (code) => {
console.log(`Child process exited with code ${code}`);
});
// Spawn with options
const grep = spawn('grep', ['hello', 'input.txt'], {
cwd: '/tmp', // Working directory
env: { ...process.env, CUSTOM_ENV: 'value' },
stdio: 'pipe', // Configure stdio
detached: false, // Process group behavior
shell: false // Whether to run in a shell
});
// Handling errors
grep.on('error', (err) => {
console.error(`Failed to start subprocess: ${err.message}`);
});
When to Use spawn()
spawn() is especially useful for:
- Long-running processes (like server processes or watchers)
- Processes that produce large amounts of output
- When you need to process data as it's generated, rather than waiting for completion
Using spawn() with stdin
const { spawn } = require('child_process');
// Spawn a process that reads from stdin
const process = spawn('wc', ['-w']); // Word count
// Send data to the process's stdin
process.stdin.write('Hello world from Node.js!');
process.stdin.end(); // Signal the end of input
// Capture output
process.stdout.on('data', (data) => {
console.log(`Number of words: ${data}`);
});
The fork() Method
The fork() method is a special case of spawn() specifically for creating Node.js processes. It sets up an IPC channel that allows sending messages between the parent and child processes.
// In the main file (parent.js)
const { fork } = require('child_process');
// Fork a child process
const child = fork('child.js');
// Send a message to the child
child.send({ message: 'Hello from parent' });
// Receive messages from the child
child.on('message', (message) => {
console.log('Message from child:', message);
});
// Handle child process exit
child.on('close', (code) => {
console.log(`Child process exited with code ${code}`);
});
// In the child file (child.js)
console.log('Child process started', process.pid);
// Listen for messages from the parent
process.on('message', (message) => {
console.log('Message from parent:', message);
// Send a message back to the parent
process.send({ response: 'Hello from child' });
// After 3 seconds, exit the process
setTimeout(() => {
process.exit(0);
}, 8080);
});
Benefits of fork()
- Each forked process gets its own V8 instance and memory
- Isolates CPU-intensive work from the main event loop
- Allows communication between processes via messages
- Helps utilize multiple CPU cores
Interprocess Communication (IPC)
Child processes created with fork() can communicate with the parent process through a built-in IPC channel using send() and the message event.
Sending Complex Data
// In parent.js
const { fork } = require('child_process');
const child = fork('worker.js');
// Send different types of data
child.send({
command: 'compute',
data: [1, 2, 3, 4, 5],
options: {
multiply: 2,
subtract: 1
}
});
// Receive the result
child.on('message', (result) => {
console.log('Computation result:', result);
child.disconnect(); // Clean up the IPC channel
});
// In worker.js
process.on('message', (msg) => {
if (msg.command === 'compute') {
const result = msg.data.map(num => num * msg.options.multiply - msg.options.subtract);
// Send the result back to the parent
process.send({ result });
}
});
Note: The messages are serialized using JSON, so you can only send JSON-compatible data (objects, arrays, strings, numbers, booleans, and null).
Managing Child Processes
Killing a Child Process
const { spawn } = require('child_process');
// Spawn a long-running process
const child = spawn('node', ['-e', `
setInterval(() => {
console.log('Still running...', Date.now());
}, 1000);
`]);
// Output from the process
child.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
// Kill the process after 5 seconds
setTimeout(() => {
console.log('Killing the child process...');
// Send a SIGTERM signal
child.kill('SIGTERM');
// Alternative: child.kill() - uses SIGTERM by default
}, 5000);
// Handle the exit event
child.on('exit', (code, signal) => {
console.log(`Child process exited with code ${code} and signal ${signal}`);
});
Detached Processes
You can create detached child processes that continue running independently of the parent:
const { spawn } = require('child_process');
const fs = require('fs');
// Create a detached process
const child = spawn('node', ['long_running_script.js'], {
detached: true,
stdio: ['ignore',
fs.openSync('output.log', 'w'),
fs.openSync('error.log', 'w')
]
});
// Unref the child to allow the parent to exit independently
child.unref();
console.log(`Started detached process with PID: ${child.pid}`);
console.log('Parent will exit while child continues running.');
// The parent can now exit, and the child will continue running
Practical Examples
Creating a Simple Task Queue
// In tasks.js (parent)
const { fork } = require('child_process');
const numCPUs = require('os').cpus().length;
class TaskQueue {
constructor() {
this.tasks = [];
this.workers = [];
this.maxWorkers = numCPUs;
}
addTask(task) {
this.tasks.push(task);
this.runNext();
}
runNext() {
// If we have tasks and available workers
if (this.tasks.length > 0 && this.workers.length < this.maxWorkers) {
const task = this.tasks.shift();
const worker = fork('worker.js');
console.log(`Starting worker for task ${task.id}`);
this.workers.push(worker);
worker.send(task);
worker.on('message', (result) => {
console.log(`Task ${task.id} completed with result:`, result);
// Remove this worker from our workers list
this.workers = this.workers.filter(w => w !== worker);
// Run the next task if we have one
this.runNext();
});
worker.on('error', (err) => {
console.error(`Worker for task ${task.id} had an error:`, err);
this.workers = this.workers.filter(w => w !== worker);
this.runNext();
});
worker.on('exit', (code) => {
if (code !== 0) {
console.error(`Worker for task ${task.id} exited with code ${code}`);
}
});
}
}
// Usage
const queue = new TaskQueue();
// Add some tasks
for (let i = 1; i <= 10; i++) {
queue.addTask({
id: i,
type: 'calculation',
data: Array.from({ length: 1000000 }, () => Math.random())
});
}
// In worker.js
process.on('message', (task) => {
console.log(`Worker ${process.pid} received task ${task.id}`);
// Simulate CPU-intensive work
let result;
if (task.type === 'calculation') {
// For example, find sum and average
const sum = task.data.reduce((acc, val) => acc + val, 0);
const avg = sum / task.data.length;
result = { sum, avg };
}
// Send result back to parent
process.send({ taskId: task.id, result });
// Exit this worker
process.exit(0);
});
Running External Applications
const { spawn } = require('child_process');
const path = require('path');
const fs = require('fs');
// Function to convert a video using ffmpeg
function convertVideo(inputFile, outputFile, options = {}) {
return new Promise((resolve, reject) => {
// Make sure input file exists
if (!fs.existsSync(inputFile)) {
return reject(new Error(`Input file ${inputFile} does not exist`));
}
// Prepare ffmpeg arguments
const args = ['-i', inputFile];
if (options.scale) {
args.push('-vf', `scale=${options.scale}`);
}
if (options.format) {
args.push('-f', options.format);
}
args.push(outputFile);
// Spawn ffmpeg process
const ffmpeg = spawn('ffmpeg', args);
// Collect output for logging
let stdoutData = '';
let stderrData = '';
ffmpeg.stdout.on('data', (data) => {
stdoutData += data;
});
ffmpeg.stderr.on('data', (data) => {
stderrData += data;
});
// Handle process completion
ffmpeg.on('close', (code) => {
if (code === 0) {
resolve({
inputFile,
outputFile,
stdout: stdoutData,
stderr: stderrData
});
} else {
reject(new Error(`ffmpeg exited with code ${code}\n${stderrData}`));
}
});
// Handle process errors
ffmpeg.on('error', reject);
});
}
// Example usage (commented out)
/*
convertVideo('input.mp4', 'output.webm', {
scale: '640:480',
format: 'webm'
})
.then(result => {
console.log('Video conversion successful!');
console.log(`Output file: ${result.outputFile}`)
;
})
.catch(error => {
console.error('Video conversion failed:', error.message);
});
*/
Best Practices
- Input Sanitization: Always sanitize user inputs to prevent command injection attacks, especially with
exec() - Resource Management: Monitor and handle the resources (memory, file descriptors) used by child processes
- Error Handling: Always have proper error handling for child processes
- Choose the Right Method:
- Use
exec()for simple commands with limited output - Use
spawn()for long-running processes or large outputs - Use
fork()for CPU-intensive Node.js operations
- Use
- Cleanup: Properly kill child processes when they're no longer needed
- Limit Concurrency: Control the number of concurrent child processes to avoid system overload
Warning: Running too many child processes can quickly exhaust system resources. Always implement rate limiting and concurrency control.
Security Considerations
- Command Injection: Never pass unsanitized user input directly to
exec()orspawn() - Environment Variables: Be careful with passing environment variables to child processes
- File Access: Understand the permissions that child processes may have
- Resource Limits: Consider implementing timeouts and memory limits for child processes
