RE: Parsing Names

<snip>
How are you tokenizing a string without using %scan() or a home grown
replacement of similar design,
while getting better performance?

As I see it, no matter what you do you have to compare each byte of the
string, so the big-O of the
algorithm will always be O(n).

I can't imagine that %scan() is written that poorly, but perhaps it is.

I would expect CPU usage to hit 100% parsing lots of names simply
because such an algorithm would be
CPU bound.
</snip>

Can't tell you why %Scan are such hogs other than you are scanning
through the string over and over again. Just know it is.

Had vendor several years ago who wrote a parser as a scan. Took our
whole system to its knees every time it ran. I ended up working for the
vendor and wrote as a parser and dropped CPU to nothing. Didn't even
know it was running.

I have written a lot of parsers using State Transition Diagrams and have
never seen any kind of performance hit.

The performance hit with scans was also just reported by someone else on
this forum recently.

Did my parser a little different than I have seen others written. If you
look at Java or VB, they return an array of strings. Trying to do that
in ILE is tough. Instead I just loop through the string getting the
location of the separator and build an array of records with the
starting position and length of the string so when you make a call
string STGF_GetToken(1), it just uses the value into the array and
returns that piece. In other words, it uses the power of ILE. That way
the function can be on the right or the left side of the equation.

We use it all over the place and have never seen a performance hit.