Frank Buschmann, Regine Meunier, Hans Rohnert, Peter Sommerlad, and Michael Stal
"The Pipes and Filters architectural pattern provides a structure for systems that process a stream of data. Each processing step is encapsulated in a filter component. Data is passed through pipes between adjacent filters. Recombining filters allows you to build families of related systems." (Buschmann, F., R. Meunier, H. Rohnert, P. Sommerlad, and M. Stal. Pattern-Oriented Software Architecture: A System Of Patterns. West Sussex, England: John Wiley & Sons Ltd., 1996)
The Pipes and Filters architectural pattern divides the task of a system into several sequential processing steps. These steps are connected by the data flow through the system--the output data of a step is the input to the subsequent step. Each processing step is implemented by a filter component. A filter consumes and delivers data incrementally--in contrast to consuming all its input before producing any output--to achieve low latency and enable real parallel processing. The input to the system is provided by a data source such as a text file. The output flows into a data sink such as a file, terminal, animation program and so on. The data source, the filters and the data sink are connected sequentially by pipes. Each pipe implements the data flow between adjacent processing steps. The sequence of filters combined by pipes is called a processing pipeline. For a more detailed Motivation description for this pattern see: Buschmann, F., R. Meunier, H. Rohnert, P. Sommerlad, and M. Stal. Pattern-Oriented Software Architecture: A System Of Patterns. West Sussex, England: John Wiley & Sons Ltd., 1996.
This pattern has been used on the following systems: UNIX made the Pipes and Filters pattern popular with the command shells and filter programs. (Bach, M.J., The Design of the UNIX Operating System, Prentice Hall, 1986.) CMS Pipelines extends the operating systems of the IBM mainframes thus supporting the Pipes and Frame architectures. (Hartmann, J., C. Reichetzeder, and M. Varian, CMS Pipelines, http://www.akhwien.ac.at/pipeline.html LASSPTools is a toolset for numerical analysis and graphics. LASSPTools contains filter programs that may be combined using UNIX pipes. (Sethna, J., LASSPTools: Graphical and Numerical Extensions to Unix, http://www.lassp.cornell.edu/LASSPTools/LASSPTools.html.)
Layers pattern
Layers pattern, Pipes and Filters pattern, Buschmann patterns, architectural patterns, data stream, filter, parallel processing, families of related systems
graphicsnumerical analysisoperating systemsprocessing data streams
different sources of input data exist (i.e., network connection or hardware sensor)frequent change requests/requirements changeglobal system task decomposes naturally into several processing stagesneed a structure for processing streams of dataneed various ways to present or store final resultsnon-adjacent processing steps do not share informationprocessing order and steps can changesequential stage problems may not relate well with interactive systemssmall processing steps are better than large processing stepsthere is significant data flow between the stagesusers need to directly alter the processing order and steps
efficiency by parallel processingflexibility by filter exchangeflexibility by recombinationno intermediate files necessary, but possiblerapid prototyping of pipelinesreuse of filter components
data transformation overheadefficiency gain by parallel processing is often an illusionerror handlingsharing state information is expensive or inflexible
Test driver of simple implementation. testpipes.cppClass implementations of simple pattern. pipesandfilters.cppClass declarations for the pattern blueprint. pipesandfilters.hExceptions thrown by pipe and filter blueprint classes. filterexceptions.h