Hartmann pipeline

Pipelines

Paradigm	Dataflow programming
Designed by	John P. Hartmann (IBM)
Developer	IBM
First appeared	1986
Stable release	1.1.12/000D / 2016-11-11
Platform	IBM z Systems
OS	z/VM 6.4
Website	http://vm.marist.edu/~pipeline
Influenced by
Pipeline (Unix)

CMS Pipelines implements the pipeline concept under the VM/CMS operating system. The programs in a pipeline operate on a sequential stream of records. A program writes records that are read by the next program in the pipeline. Any program can be combined with any other because reading and writing is done through a device independent interface.

CMS Pipelines provides a CMS command, PIPE. The argument string to the PIPE command is the pipeline specification. PIPE selects programs to run and chains them together in a pipeline to pump data through.

Because CMS programs and utilities don't provide a device independent stdin and stdout interface, CMS Pipelines has a built-in library of programs that can be called in a pipeline specification. These built-in programs interface to the operating system, and perform many utility functions.

Data on CMS is structured in logical records rather than a stream of bytes. For textual data a line of text corresponds to a logical record. In CMS Pipelines the data is passed between the stages as logical records.

CMS Pipelines users issue pipeline commands from the terminal or in EXEC procedures. Users can write programs in REXX that can be used in addition to the built-in programs.

A simple example that reads a disk file, separates records containing the string "Hello" from those that do not. The selected records are modified by appending the string "World!" to each of them; the other records are translated to upper case. The two streams are then combined and the records are written to a new output file.

In this example, the < stage reads the input disk file and passes the records to the next stage in the pipeline. The locate stage separates the input stream into two output streams. The primary output of locate (records containing Hello) passes the records to the insert stage. The insert stage modifies the input records as specified in its arguments and passes them to its output. The output is connected to faninany that combines records from all input streams to form a single output stream. The output is written to the new disk file.

The secondary output of locate (marked by the second occurrence of the a: label) contains the records that did not meet the selection criterion. These records are translated to upper case (by the xlate stage) and passed to the secondary input stream of faninany (marked by the second occurrence of the i: label).