FilterBase

Signature: tokens → tokens

All provided filters (pick, replace, ignore, filter) are built on filterBase. It is a factory that produces configurable token-stream filter functions.

filterBase operates on token streams produced by a parser or another filter. Filters go after a parser and can be chained.

API

This document describes the user-facing interface only. If you want to build your own filter, feel free to inspect the code to gain more insights.

Internally, FilterBase keeps track of objects by building a stack. Items of a stack can be:

• Number. In this case, a corresponding object is an array, and the number is the current index.
• String. In this case, a corresponding object is an object, and the string is the current property key.
• null. In this case, a corresponding object is an object, but keys are not tracked. FilterBase keeps track of keys only if a previous stream returns packed keys. In this case, a filter assumes that only object's shape will be used for filtering.

The stack is used to make filtering.

Options

All filter factories accept an options object with the following properties:

• pathSeparator is a string that separates stack values when it is converted to a string. The algorithm is straightforward: stack.join(pathSeparator). The default: '.'.

  const obj = [{a: 1}, {b: 2}];
  
  // stack when filtering 1: [0, 'a']
  // converted to a string: '0.a'
  
  // stack when filtering 2: [1, 'b']
  // converted to a string: '1.b'

• filter is a way to accept or reject a data item. The interpretation of its returned value is up to concrete filter objects. Its value can be one of the following types:

  • String. The stack is converted to a string using pathSeparator, then it should be equal to filter value, or it should be longer and the filter value should be on a boundary of the pathSeparator value.

    const obj = {a: [1, 2], ab: null};
    
    const filter = 'a';
    // it fits ['a'], ['a', 0], and ['a', 1], but not ['ab']

  • RegExp. The stack is converted to a string using pathSeparator, then the filter is applied using filter.test(path).

    const obj = {a: [1, 2], ab: null};
    
    const filter = /^a\b/;
    // it fits ['a'], ['a', 0], and ['a', 1], but not ['ab']
    
    const filter = /^a/;
    // it fits ['a'], ['a', 0], ['a', 1], and ['ab']

  • Function. It is called as filter(stack, chunk), where chunk is a data item being filtered. It returns a truthy or falsy value.

  • The default: () => true.

• once is a flag. When it is truthy, a filter object will make a selection (depending on its definition of selection) only once. Otherwise, all selections are included. The default: false.

  • It can be used as an optimization feature when we know that our stream contains exactly one object we want to do our action on.

• maxDepth is the maximum JSON nesting depth a filter evaluates. When a token is nested deeper than this, the filter throws a RangeError instead of matching its path — a guard for untrusted input with unbounded nesting. The default: 1024. Pass Infinity to remove the limit.

• replacement is what should be used instead of skipped objects. Not all filters use this option. Its value can be one of the following types:

  • Function. It is called as replacement(stack, chunk, options), where chunk is a data item being filtered. Its result is interpreted like the static values below; none (from stream-chain) removes the value.
  • An array of tokens, or a single token object, forming a semantically valid JSON value (see Parser).
  • Any other JavaScript value — a number, string, boolean, null, array, or plain object — is disassembled into tokens once and substituted as that JSON value, shaped by the same packing/streaming options as the parser. An empty array is an empty token list and removes the value (kept for compatibility); for an empty JSON array pass [{name: 'startArray'}, {name: 'endArray'}].
  • The default for replace: none (the value is removed). Specify replacement explicitly to substitute with a different value.

• Key replay: when a filter recreates a parent object (see makeStackDiffer), it replays the parent's key in the forms it received it. keyValue is always replayed, because a key is tracked only when it arrived packed; the streamed form (startKey, stringChunk, endKey) is replayed when streamed keys have been received from upstream.

  • streamValues seeds streamKeys; streamKeys overrides the mirroring: true always replays the streamed form, false never does.
  • packKeys is accepted for compatibility but has no effect on a filter; in a withParser() options bag it still configures the parser. Deprecated; to be removed in the next major.

• Input requirement: property keys are tracked on the stack only when they arrive as keyValue tokens (see Stack and path); unpacked (streamed-only) keys are not tracked, so key-based paths cannot match and parents cannot be recreated. Parser packs keys by default; keep packKeys: true on it when you disable packValues (withParser() and its variants set it for you).

Important details

Stack and path

When using a string or a regular expression as a filter function, the stack is converted to a path string before the filter can be applied. It should be noted that when a source stream does not produce keyValue data items, the stack uses null to denote an undefined property key, which is converted to a path string as an empty string:

[].join('.'); // ''
[null].join('.'); // ''
[null, null].join('.'); // '.'
[null, 1, null].join('.'); // '.1.'
[1, null, null, null, 2, null].join('.'); // '1....2.'

Be aware of this behavior when crafting filters.

Property keys can be arbitrary strings. Sometimes it can mess up paths and textual filters. In order to avoid it, you can choose a different pathSeparator. It can be any string you like, just make sure it works with your filters.

const f = filter({pathSeparator: '->'});
// it will produce paths like that:
// [1, 'a'] => '1->a'
// [1, 0, 'ab', 0] => '1->0->ab->0'

Replacement hazards

Filters do not check if an array of replacement items is valid or not. Malformed arrays will produce substreams, which can break the rest of the data pipeline. Be extra careful with the replacement option.

makeStackDiffer

Named export from stream-json/filters/filter-base.js. Returns a function that emits the structural tokens (startObject, startArray, startKey + stringChunk + endKey, endObject, endArray) required to reconstruct the surrounding JSON envelope between two stack positions. Used internally by filter and replace to bridge non-contiguous matches; exposed for consumers building custom filters on top of filterBase.

import {makeStackDiffer} from 'stream-json/filters/filter-base.js';

const differ = makeStackDiffer(/* previousStack */ []);
// differ(stack, chunk, options) → Many<Token>
// Call it on each match site; the returned tokens carry the structural
// envelope (open object/array, key tokens) needed to land `chunk` at
// the right depth in the output stream.

Arguments:

• previousStack (optional) — initial stack the differ should treat as the "already emitted" position. Defaults to [] (root). Pass the stack state from the previous emit when chaining diffs.

The returned function takes (stack, chunk, options) and returns a Many<Token> of the bridging structural tokens, where options is the filter's FilterBaseOptions (so the differ honors streamKeys, streamValues, and pathSeparator; it always replays keyValue).

This is a low-level utility — most users compose existing filters (pick, replace, ignore, filter) instead. Reach for it when writing a custom filter on top of filterBase that needs to recreate parent containers between non-adjacent matches.