Re: Question Re: Piping and Redirection

Hans is correct, but I'd like to re-word a bit.  In unix (and in fact in C
and other languages), files can be opened as text streams or binary files.
stdin and stdout are always expected to be text streams (though that is not
a strict nor enforced requirement).  Usefulness of binary data for piping
purposes though, would have to be questionable, and can hurt you if you
pipe pipe through some filters such as more... or allow to proceed to the
display terminal.

If you have an Oracle database, for example, though, that database is
comprised of binary data, accessible randomly at the file level in all the
ways you are used to.  The results of "cat /usr/oracle/data/main.dbf"
(where that is one of the files comprising a tablespace) would be quite
similar (i.e. illegible) to the results of a binary file on the iSeries.

The point is that stdin and stdout (and stderr, which we hadn't really
brought into this discussion) stream only text data by design, because the
utilites (filters) that we run them through are built to operate on that
sort of data.  (Or do we build the utilities in such a way because the data
is text-based?  Nah, it's the former.)

cat, sed, awk, grep, more... all those others don't play well in that sort
of environment.  For those, you have applications that understand the data
at a structured level.

Dennis





Hans Boldt <boldt@ca.ibm.com>@midrange.com on 11/19/2002 08:39:41 AM

Please respond to midrange-l@midrange.com

Sent by:    midrange-l-admin@midrange.com


To:    midrange-l@midrange.com
cc:
Subject:    Re: Question Re: Piping and Redirection


Tom Liotta wrote:
> To the *nix-heads:
>
> With all the back and forth that was going on over which command line is
better, I'd like to ask a question about the piping and redirection on the
*nix side of things. I'm familiar with the concepts, but I'm not clear on
how the issue of non-streamfile manipulation would be handled by various
*nix shells.
>
> That is, under QSHELL, I can pretty much pipe the output of one command
into another just as I could under just about any other *nix shell. And if
I don't have a particular utility, I can build my own or possibly port one
so that I'll have the same utilities that are available elsewhere.
Disregarding any fancy facilities some great shell might have beyond just
piping and redirection, on other systems, how do basic commands/utilities
such as sed and grep handle files that aren't streamfiles?
>
> That is, traditional OS/400 commands commonly wouldn't support something
like piping because they generally rely on a database record format. Is it
even meaningful to make comparisons outside of QSHELL?


Strictly speaking, in Unix everything is a stream file.

It's up to particular programs and applications to assign specific
meanings to the bytes that make up a file. And so, if you want to
process certain types of files intelligently, you need specific
application programs. For certain types of collections of data, like
databases and images, processing that data using a stream model
makes little sense. For example, you wouldn't want to use grep on a
database file. Instead, you'd open an SQL command prompt and search
the database using a SELECT statement. Likewise, if you wanted to
scale an image file, you wouldn't use something like sed or perl,
you'd use an X based image program, like the gimp, or a command line
utility like ImageMagick.

On the other hand, the stream model works very nicely for files
involved with programming, such as program source files and scripts,
make files, html documents, etc.