Streaming Operation

HyperScan's simplest use-case is a block scanning application. Such an application merely wants to scan a single contiguous block of data with a set of regular expressions and collect any matches that occur. For these cases, HyperScan provides a block mode interface which stores no state and returns all of the matches before it completes.

Many applications operate on data that may not be available as a single block. For example, network traffic scanning applications often cannot hold in memory all of the packets that make up a message, and simply scanning each packet ignores matches that straddle packet boundaries. To support those cases, HyperScan also provides a streaming API, enabling such applications to easily implement cross-packet inspection. 

In streaming mode, the application can pass HyperScan a stream of data blocks, one at a time, and HyperScan will return matches as they occur, even matches that cross the boundaries between these blocks.

Streaming support is a first-class citizen for HyperScan: matching is supported across an arbitrary number of block writes, and the full complement of supported PCRE constructs can be used.

Streaming operation requires a small fixed-size stream record to store the state associated with each stream, and HyperScan provides an easy-to-use set of interfaces for manipulating these records.