在 Node.js v12.16.2 版本中，`Buffer` 是用于处理二进制数据的核心模块之一

Node.js v12.16.2 中 Buffer 的文档概述

在 Node.js v12.16.2 版本中，Buffer 是用于处理二进制数据的核心模块之一。它提供了一种机制来创建和操作原始字节序列，这使得开发者可以轻松地读取文件、处理网络流以及执行其他涉及二进制数据的操作。

创建 Buffer 实例的方法

可以通过多种方式创建一个新的 Buffer 对象。以下是几种常见的方法：

• 使用静态方法 Buffer.from() 来从数组或其他输入源初始化缓冲区。

  const buf = Buffer.from([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]);
  console.log(buf); // 输出: <Buffer 00 01 02 03 04 05 06 07 08 09>
  

• 利用 Buffer.allocUnsafe(size) 方法分配未清零的内存空间。需要注意的是，这种方法不会自动填充新分配的空间为零值，因此可能包含垃圾数据。

  const unsafeBuf = Buffer.allocUnsafe(10);
  console.log(unsafeBuf); // 可能输出类似于: <Buffer e8 f4 a3 d0 ... >
  

• 调用 Buffer.alloc(size[, fill][, encoding]) 函数显式指定大小并可选填充值及编码格式。

  const filledBuf = Buffer.alloc(10, 'a');
  console.log(filledBuf.toString()); // 输出: "aaaaaaaaaa"
  

关键属性与行为特性

• 默认池尺寸 (Buffer.poolSize)：为了优化性能，在内部实现上存在一个固定大小 (通常为 8KB 或者更具体地说是 8192 字节) 的共享内存池用来管理小型 Buffers 的分配过程。如果请求创建的小于等于该阈值，则会尝试复用此预定义区域内的资源；反之则单独开辟新的堆外存储位置。

• 不可变性提醒：尽管可以直接修改已存在的 Buffer 数据内容，但从安全角度考虑建议尽可能保持其不变状态以减少潜在错误风险。

常见 API 功能列表

除了上述提到的基础构建手段之外，还有许多实用的功能可供调用比如但不限于字符串转换、切片提取子集等功能均被广泛应用于实际项目开发当中去解决各类需求场景下的挑战。

// 示例展示部分常用API功能演示代码片段如下所示：
const exampleStr = 'hello world';
let strToBuf = Buffer.from(exampleStr);

console.log(strToBuf.toString('base64'));       // base64 编码形式表示原串："aGVsbG8gd29ybGQ="
console.log(Buffer.concat([strToBuf]));        // 合并多个buffer成单个连续体
console.log(strToBuf.slice(0, 5).toString());  // 提取前五个字符作为新对象返回:"hello"

Node.js v12.16.2 Documentation Buffer
Table of Contents

Buffer

    Buffer.from(), Buffer.alloc(), and Buffer.allocUnsafe()
        The --zero-fill-buffers command line option
        What makes Buffer.allocUnsafe() and Buffer.allocUnsafeSlow() "unsafe"?
    Buffers and Character Encodings
    Buffers and TypedArray
    Buffers and iteration

    Class: Buffer
        new Buffer(array)
        new Buffer(arrayBuffer[, byteOffset[, length]])
        new Buffer(buffer)
        new Buffer(size)
        new Buffer(string[, encoding])
        Class Method: Buffer.alloc(size[, fill[, encoding]])
        Class Method: Buffer.allocUnsafe(size)
        Class Method: Buffer.allocUnsafeSlow(size)
        Class Method: Buffer.byteLength(string[, encoding])
        Class Method: Buffer.compare(buf1, buf2)
        Class Method: Buffer.concat(list[, totalLength])
        Class Method: Buffer.from(array)
        Class Method: Buffer.from(arrayBuffer[, byteOffset[, length]])
        Class Method: Buffer.from(buffer)
        Class Method: Buffer.from(object[, offsetOrEncoding[, length]])
        Class Method: Buffer.from(string[, encoding])
        Class Method: Buffer.isBuffer(obj)
        Class Method: Buffer.isEncoding(encoding)
        Class Property: Buffer.poolSize
        buf[index]
        buf.buffer
        buf.byteOffset
        buf.compare(target[, targetStart[, targetEnd[, sourceStart[, sourceEnd]]]])
        buf.copy(target[, targetStart[, sourceStart[, sourceEnd]]])
        buf.entries()
        buf.equals(otherBuffer)
        buf.fill(value[, offset[, end]][, encoding])
        buf.includes(value[, byteOffset][, encoding])
        buf.indexOf(value[, byteOffset][, encoding])
        buf.keys()
        buf.lastIndexOf(value[, byteOffset][, encoding])
        buf.length
        buf.parent
        buf.readBigInt64BE([offset])
        buf.readBigInt64LE([offset])
        buf.readBigUInt64BE([offset])
        buf.readBigUInt64LE([offset])
        buf.readDoubleBE([offset])
        buf.readDoubleLE([offset])
        buf.readFloatBE([offset])
        buf.readFloatLE([offset])
        buf.readInt8([offset])
        buf.readInt16BE([offset])
        buf.readInt16LE([offset])
        buf.readInt32BE([offset])
        buf.readInt32LE([offset])
        buf.readIntBE(offset, byteLength)
        buf.readIntLE(offset, byteLength)
        buf.readUInt8([offset])
        buf.readUInt16BE([offset])
        buf.readUInt16LE([offset])
        buf.readUInt32BE([offset])
        buf.readUInt32LE([offset])
        buf.readUIntBE(offset, byteLength)
        buf.readUIntLE(offset, byteLength)
        buf.subarray([start[, end]])
        buf.slice([start[, end]])
        buf.swap16()
        buf.swap32()
        buf.swap64()
        buf.toJSON()
        buf.toString([encoding[, start[, end]]])
        buf.values()
        buf.write(string[, offset[, length]][, encoding])
        buf.writeBigInt64BE(value[, offset])
        buf.writeBigInt64LE(value[, offset])
        buf.writeBigUInt64BE(value[, offset])
        buf.writeBigUInt64LE(value[, offset])
        buf.writeDoubleBE(value[, offset])
        buf.writeDoubleLE(value[, offset])
        buf.writeFloatBE(value[, offset])
        buf.writeFloatLE(value[, offset])
        buf.writeInt8(value[, offset])
        buf.writeInt16BE(value[, offset])
        buf.writeInt16LE(value[, offset])
        buf.writeInt32BE(value[, offset])
        buf.writeInt32LE(value[, offset])
        buf.writeIntBE(value, offset, byteLength)
        buf.writeIntLE(value, offset, byteLength)
        buf.writeUInt8(value[, offset])
        buf.writeUInt16BE(value[, offset])
        buf.writeUInt16LE(value[, offset])
        buf.writeUInt32BE(value[, offset])
        buf.writeUInt32LE(value[, offset])
        buf.writeUIntBE(value, offset, byteLength)
        buf.writeUIntLE(value, offset, byteLength)
    buffer.INSPECT_MAX_BYTES
    buffer.kMaxLength
    buffer.transcode(source, fromEnc, toEnc)

    Class: SlowBuffer
        new SlowBuffer(size)

    Buffer Constants
        buffer.constants.MAX_LENGTH
        buffer.constants.MAX_STRING_LENGTH

Buffer

Stability: 2 - Stable

Prior to the introduction of TypedArray, the JavaScript language had no mechanism for reading or manipulating streams of binary data. The Buffer class was introduced as part of the Node.js API to enable interaction with octet streams in TCP streams, file system operations, and other contexts.

With TypedArray now available, the Buffer class implements the Uint8Array API in a manner that is more optimized and suitable for Node.js.

Instances of the Buffer class are similar to arrays of integers from 0 to 255 (other integers are coerced to this range by & 255 operation) but correspond to fixed-sized, raw memory allocations outside the V8 heap. The size of the Buffer is established when it is created and cannot be changed.

The Buffer class is within the global scope, making it unlikely that one would need to ever use require(‘buffer’).Buffer.

// Creates a zero-filled Buffer of length 10.
const buf1 = Buffer.alloc(10);

// Creates a Buffer of length 10, filled with 0x1.
const buf2 = Buffer.alloc(10, 1);

// Creates an uninitialized buffer of length 10.
// This is faster than calling Buffer.alloc() but the returned
// Buffer instance might contain old data that needs to be
// overwritten using either fill() or write().
const buf3 = Buffer.allocUnsafe(10);

// Creates a Buffer containing [0x1, 0x2, 0x3].
const buf4 = Buffer.from([1, 2, 3]);

// Creates a Buffer containing UTF-8 bytes [0x74, 0xc3, 0xa9, 0x73, 0x74].
const buf5 = Buffer.from(‘tést’);

// Creates a Buffer containing Latin-1 bytes [0x74, 0xe9, 0x73, 0x74].
const buf6 = Buffer.from(‘tést’, ‘latin1’);

Buffer.from(), Buffer.alloc(), and Buffer.allocUnsafe()

In versions of Node.js prior to 6.0.0, Buffer instances were created using the Buffer constructor function, which allocates the returned Buffer differently based on what arguments are provided:

Passing a number as the first argument to Buffer() (e.g. new Buffer(10)) allocates a new Buffer object of the specified size. Prior to Node.js 8.0.0, the memory allocated for such Buffer instances is not initialized and can contain sensitive data. Such Buffer instances must be subsequently initialized by using either buf.fill(0) or by writing to the entire Buffer. While this behavior is intentional to improve performance, development experience has demonstrated that a more explicit distinction is required between creating a fast-but-uninitialized Buffer versus creating a slower-but-safer Buffer. Since Node.js 8.0.0, Buffer(num) and new Buffer(num) return a Buffer with initialized memory.
Passing a string, array, or Buffer as the first argument copies the passed object's data into the Buffer.
Passing an ArrayBuffer or a SharedArrayBuffer returns a Buffer that shares allocated memory with the given array buffer.

Because the behavior of new Buffer() is different depending on the type of the first argument, security and reliability issues can be inadvertently introduced into applications when argument validation or Buffer initialization is not performed.

For example, if an attacker can cause an application to receive a number where a string is expected, the application may call new Buffer(100) instead of new Buffer(“100”), it will allocate a 100 byte buffer instead of allocating a 3 byte buffer with content “100”. This is commonly possible using JSON API calls. Since JSON distinguishes between numeric and string types, it allows injection of numbers where a naive application might expect to always receive a string. Before Node.js 8.0.0, the 100 byte buffer might contain arbitrary pre-existing in-memory data, so may be used to expose in-memory secrets to a remote attacker. Since Node.js 8.0.0, exposure of memory cannot occur because the data is zero-filled. However, other attacks are still possible, such as causing very large buffers to be allocated by the server, leading to performance degradation or crashing on memory exhaustion.

To make the creation of Buffer instances more reliable and less error-prone, the various forms of the new Buffer() constructor have been deprecated and replaced by separate Buffer.from(), Buffer.alloc(), and Buffer.allocUnsafe() methods.

Developers should migrate all existing uses of the new Buffer() constructors to one of these new APIs.

Buffer.from(array) returns a new Buffer that contains a copy of the provided octets.
Buffer.from(arrayBuffer[, byteOffset[, length]]) returns a new Buffer that shares the same allocated memory as the given ArrayBuffer.
Buffer.from(buffer) returns a new Buffer that contains a copy of the contents of the given Buffer.
Buffer.from(string[, encoding]) returns a new Buffer that contains a copy of the provided string.
Buffer.alloc(size[, fill[, encoding]]) returns a new initialized Buffer of the specified size. This method is slower than Buffer.allocUnsafe(size) but guarantees that newly created Buffer instances never contain old data that is potentially sensitive. A TypeError will be thrown if size is not a number.
Buffer.allocUnsafe(size) and Buffer.allocUnsafeSlow(size) each return a new uninitialized Buffer of the specified size. Because the Buffer is uninitialized, the allocated segment of memory might contain old data that is potentially sensitive.

Buffer instances returned by Buffer.allocUnsafe() may be allocated off a shared internal memory pool if size is less than or equal to half Buffer.poolSize. Instances returned by Buffer.allocUnsafeSlow() never use the shared internal memory pool.
The --zero-fill-buffers command line option

Added in: v5.10.0

Node.js can be started using the --zero-fill-buffers command line option to cause all newly-allocated Buffer instances to be zero-filled upon creation by default. Without the option, buffers created with Buffer.allocUnsafe(), Buffer.allocUnsafeSlow(), and new SlowBuffer(size) are not zero-filled. Use of this flag can have a significant negative impact on performance. Use the --zero-fill-buffers option only when necessary to enforce that newly allocated Buffer instances cannot contain old data that is potentially sensitive.

$ node --zero-fill-buffers

Buffer.allocUnsafe(5);
<Buffer 00 00 00 00 00>

What makes Buffer.allocUnsafe() and Buffer.allocUnsafeSlow() “unsafe”?

When calling Buffer.allocUnsafe() and Buffer.allocUnsafeSlow(), the segment of allocated memory is uninitialized (it is not zeroed-out). While this design makes the allocation of memory quite fast, the allocated segment of memory might contain old data that is potentially sensitive. Using a Buffer created by Buffer.allocUnsafe() without completely overwriting the memory can allow this old data to be leaked when the Buffer memory is read.

While there are clear performance advantages to using Buffer.allocUnsafe(), extra care must be taken in order to avoid introducing security vulnerabilities into an application.
Buffers and Character Encodings

History

When string data is stored in or extracted out of a Buffer instance, a character encoding may be specified.

const buf = Buffer.from(‘hello world’, ‘ascii’);

console.log(buf.toString(‘hex’));
// Prints: 68656c6c6f20776f726c64
console.log(buf.toString(‘base64’));
// Prints: aGVsbG8gd29ybGQ=

console.log(Buffer.from(‘fhqwhgads’, ‘ascii’));
// Prints: <Buffer 66 68 71 77 68 67 61 64 73>
console.log(Buffer.from(‘fhqwhgads’, ‘utf16le’));
// Prints: <Buffer 66 00 68 00 71 00 77 00 68 00 67 00 61 00 64 00 73 00>

The character encodings currently supported by Node.js include:

'ascii': For 7-bit ASCII data only. This encoding is fast and will strip the high bit if set.

'utf8': Multibyte encoded Unicode characters. Many web pages and other