Python Pipelines
From Wikipedia, the free encyclopedia
Python Pipelines is a cross-platform implementation of Hartmann pipeline, written in Python.
Pipelines lets you solve a complex problem by breaking it up into a series of smaller, less complex programs. These simple programs, also called stages, can then be hooked together to get the results you want. The output resulting from a stage is the input to the next stage. A series of stages is called a pipeline. Each stage consists of a stage and its operands. Pipelines has many built-in stages; you may add your own written in Python or pseudo-Rexx.
Stages may have more than one input and/or output stream; these streams may be connected to other stages in no particular order.
Pipelines is a superset of pipes as found in Unix shells.
Python Pipelines may be invoked from a command prompt or incorporated into another Python program as an imported module.
In CMS Pipelines users may write their own stages using the Rexx programming language. Due to the nature of the 'dispatcher' (each stage is a co-routine) these program are more complex than they could be.
Python Pipeline supports coding the IBM way (in 'pseudo-Rexx'), which makes it somewhat easy to incorporate legacy code or write new stages using a known technology.
Ideally, however, Python Pipeline users learn to write their own stages in Python, using a much simpler approach (each stage has a run() method that is invoked as many times as needed). Seven or more lines of Rexx are thus replaced by two lines of Python.
[edit] Examples
Invoking the program from a command prompt:
python pipe.py "< input.txt | A: locate /Hello/ | > found.txt ; A: | > notfound.txt"
A minimal REXX Stage - just passes each input record to the output:
/* PASSRECORD REXX */
signal on error
do forever
'PEEKTO record'
/* process and/or test self.record and/or test self.RC */
'OUTPUT' record
'READTO'
end
error:
exit (RC * (RC /= 12 & RC /= 8))
The equivalent in Python:
import stage
class PassRecord(stage.Stage):
def run(self):
self.output(self.record)
