RE: Sorting file based on variable string

Hi, Vinay:

Oh, I get it. Youch.

So, one solution would be to put your key field at the front with the rest
of the record intact, then sort by the entire record, and then strip the
extra stuff we put on the front. (Hold your breath for a visual, up
shortly.)

Yeah, we can do that. By the way, I'm assuming your X'1C' is the character
we'll see after translation to ASCII (or that it's already an ASCII file).
If not, then you'll need minor adjustments here; you'll see where.

We'll first pipe through sed (could use awk, but that's not available under
QSH). This will find the field between x'1c6532' (x'1c' + 'A' + '2') and
the immediately following x'1c'. Then it will put that field on the front
of the entire record (so that that field will occur twice: once at the
front, and in its original place). Using , instead of x'1c' for visible
demonstration, take this:

John,Adam,A2Smith,1 any street,somewhere

Sam,A2Jones,2 some street,mycity,nj,07041

Mary,Elouise,Jessica,A2Doe,,Hendersonville,va

<<< --- Our sed output will be: --->>>

Smith,John,Adam,A2Smith,1 any street,somewhere

Jones,Sam,A2Jones,2 some street,mycity,nj,07041

Doe,Mary,Elouise,Jessica,A2Doe,,Hndersonville,va



Now we can sort this easily (sort < infile > outfile) and then strip off the
extra field on the front.

So, here we go. The first sed line would look like:

sed 's%\(.*\x1cA2\)\([^\x1c]*\)\(.*\)%\2\x1c\1\2\3%' < sortme.in

This saves everything up to & including the x'1c' followed by A2 into
buffer1, everything after A2 until & including the next x\1c into buffer2,
and everything else into buffer3. It then outputs: buffer2 x'1c'
buffer1 buffer2 buffer3
(spaces are for visual clarity only; they won't be in the output)

And now we can sort this by piping the output through sort

| sort

And now we pipe through sed to strip off the extra field on the front of the
"records" to provide our sorted output:

| sed 's%[^\x1c]*\x1c%%' > sorted.out

This takes everything that's not x'1c' and the first x'1c' and lops 'em
off... then writes the rest to sorted.out.

So now our sequence would look like:


CPYTOSTMF ... TOSTMF('sortme.in')
QSH CMD('sed "s%\(.*\x1cA2\)\([^\x1c]*\)\(.*\)%\2\x1c\1\2\3%" < sortme.in |
sort | sed "s%[^\x1c]*\x1c%%" > sorted.out') CPYFRMSTMF ...
FROMSTMF('sorted.out')

Aside from the difficult you might have with x'1c', I think this'll do it
for you.

Dennis Lovelady
http://www.linkedin.com/in/dennislovelady
--
"A doctor can bury his mistakes but an architect can only advise his client
to plant vines."
-- Frank Lloyd Wright


On Behalf Of Vinay Gavankar

Thanks Dennis.

I was afraid that when I simplified my requirement, I would miss something

in

the translation.

Actually, the key I want to sort with need not be the second field

separated

by x'1C'. It is not the first, it is not the last, but could be anywhere

in

between. The only uniqueness is that it starts with x'1C'A2 and ends with
another x'1C', which is the beginning of the next field. (A2 is the field

tag

that identifies the field I need).

So, based on your explanation of what the Unix sort does, your solution

might

need some tweaking, but I have no idea what it would be.

On Sat, Jan 21, 2012 at 4:04 PM, Dennis Lovelady

<iseries@xxxxxxxxxxxx>wrote:

Hi, Vinay:

I would use the unix sort command for this.

Use CPYTOSTMF to put the file on the IFS. Then use QSH "sort" command.
Then use CPYFRMSTMF to put it back

For example:
CPYTOSTMF FROMMBR('/qsys.lib/mylib.lib/myfile.file/sortme.mbr')

TOSTMF('/tmp/sortme.in')

qsh cmd('sort -t x''1c'' -k2 < /tmp/sortme.in > /tmp/sorted.out')

CPYFRMSTMF FROMSTMF('/tmp/sorted.out')
TOMBR('/qsys.lib/mylib.lib/myfile.file/sorted.mbr')



Above will
take member SORTME of file MYLIB/MIFILE, put it onto the /tmp
directory, named sortme.in
sort /tmp/sortme.in by the second "key", where the keys are
separated by x'16'
producing /tmp/sorted.out
copy sorted.out inot member SORTED of MYLIB/MYFILE


Dennis Lovelady
http://www.linkedin.com/in/dennislovelady
--
"This is the most extraordinary collection of talent, of human
knowledge, that has ever been gathered together at the White House --
with the possible exception of when Thomas Jefferson dined here
alone."
-- John F. Kennedy (to Nobel Prize winners)

I have a flat file, which has fields delimited by field separators
and a

2

character field tag. The actual fields are variable length, so not
in a

fixed

position.

For example, my field separator is hex '1C' and the field tags are
A1

thru
A6.

Tag A1 denotes First name, A2 Last name, A3 Address Line 1, A4
Address

Line 2

etc.

I want to sort by last name, which will be a variable length field,
and

could

occur anywhere in the record (as the First name is also of variable

length).

It will always be enclosed between hex '1C'A2 and hex '1C' (it will
never

be

the first or the last field in the record).

What would be the best way of doing this? (The only solution I have
is to write a program to create an intermediate file, with the Last
Name as a separate field).

The file is actually 4000 bytes and has about 18 million records,
but

this

would be a one time effort.

TIA
Vinay
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L)
mailing

list To

post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe,

or

change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please
take a

moment

to review the archives at http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list To post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take
a moment to review the archives at
http://archive.midrange.com/midrange-l.

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list To

post a message email: MIDRANGE-L@xxxxxxxxxxxx To subscribe, unsubscribe,

or

change list options,
visit: http://lists.midrange.com/mailman/listinfo/midrange-l
or email: MIDRANGE-L-request@xxxxxxxxxxxx Before posting, please take a

moment

to review the archives at http://archive.midrange.com/midrange-l.