Re: recommendations for faster IFS and string handling sub-procs. -- RPG400-L

Scott,

see comments in-line.
------original message--------
> > in the tutorial, you used a pointer field instead of the really big
> > varying options(*varsize) field for the returned record.


> I used varying fields on Friday night because that's what your
application
> seemed to call for?

i figured that, but I was just guessing at using the varying fields.

> > looking at the two examples, it almost looks to me that it's like six
of
> > one, half a dozen of another.  in the first case, you are only passing
> > pointers and integers - very fast.

> In both cases you're only passing pointers and integers.  When you pass
> a variable by reference (as I was doing in my recommended prototype with
> the varying string) all that's being passed is a pointer that that
> variable.

I get so confused with that - by NOT specifying 'value', you are passing by
reference?

> > but once you have the return values, you are still dealing with a field
> > that is sized as the maximum size you could recieve, with the trimming
> > overhead that implies.

> That's not true.    In both cases, you're passed back a "length".   In my
> example from the IFS tutorial, the length is the return value of the
> procedure...   in yours, the length is the first 2 bytes of the varying
> string.   In either case, there's a length there that you can use to
> say "ignore anything after this point.

right - i see that now.

> The overhead in trimming is when you don't know the length, and you have
> to scan through the contents of the variable until you find the last
> non-blank character (that's what %trim does).   It's unnecessary when you
> have a length variable that tells you...

> The advantage to my original approach is that it sets no absolute limit
> on the size of the data you're storing the string into.  The one with
> the varying string imposes a 32k limit.

that's a limit i'm willing to live with, for now...

> The advantage to the varying string approach is that it's easier to use
> for the pointer-phobic world of RPG programmers, and since the length is
> self-contained within the string, the string is probably ready-to-use
when
> the procedure exits.

> > in the second, you are sending the entire size of the maximum field
size,
> > but once you get it back, it's as small as the record size itself.

> I'm not sure what this is trying to say. :)

ignore that last statement.  you answered it already.

> > but after thinking about it, since the first example returns the record
> > length, it seems that it might be faster - once recieved, you could do
> > this:

> > D Record                 32765A   varying
> > D Line                   32765A
> >
> > c               dow     1 = 1
> > c               eval    reclen = readline(fd: %addr(line): %size(line))
> > c               if      reclen < 1
> > c               leave
> > c               end
> > c               eval    Record = %substr(line:1:reclen)
> > -------> process record
> > c               enddo
> > and then you are working with a varing string, only the size of the
actual
> > record.

> True, but now you've made it much slower, because you're copying the data
> from one variable to another.   It's faster than your original example
> that you posted, since it's only copying the data that it NEEDS, not the
> entire 32k.   But it's slower than reading the data into the varying
field
> in the first place.

but, since the read() or fgets() doesn't use varying strings, you are still
going to have to move the data from a fixed length (pointer based) field to
a varying length field anyway, right?   unless you can use varying fields
in read() or fgets().  (don't answer this yet - I answer it myself later
on...)

> > even if I set the maxlen to 32765, i'm not passing that data back and
> > forth, and I immediatly reduce the field to a managable size.
> > what do you think?

> But you are passing it back and forth...   your copying it from "line"
> to "record".   It's the copying of the string from one area of memory to
> another that takes the time.

see above

> As I said, this is better than your original example, since you're only
> copying the data you NEED instead of the whole string.   But it's slower
> than reading the data directly into the variable you want it in.

without a doubt, either method is better than my original example.  damned
near anything would be better.

> But, you COULD do that with "readline":

>   c               eval    %len(record) = 32765
>   c               eval    reclen = readline(fd: %addr(record)+2:
>   c                          %len(record))
>   c               if      reclen < 1
>   c               leave
>   c               end
>   c               eval    %len(record) = reclen;

> I haven't tested this code, I'm just making it up as part of the text of
> this message -- but, in theory, you're only copying the data one time,
> from the file itself to the variable.   then, you're just setting the
> value of an integer when you set the length.

I see - the read() and or fgets() doesn't need to allow varying strings.
the +2 on the address takes care of that.

I like that idea - memory is allocated for whatever I need, and no large
amounts of data are passed by reference or eval.

it's all a little less than hazy now.   time to put it all together.

> > > I'm always changing the way I do things, constantly trying different
> > > ways and different ideas, hoping to improve myself.
> >
> > me too, although i rarely improve my techniques for processing speed,
only
> > for coding speed and readability.  I hate looking at programs I wrote
as
> > little as a year ago - i'm like... 'what the heck was I thinking?'

> Hehe... I think we all do that.

I can't wait to look at this stuff in a year.  should be good for a laugh.

thanks again

rick