Re: Find/Replace in IFS file - The fastest way to do it

On Thu, 2005-10-20 at 15:31 -0500, albartell wrote:
> Hi  y'all,
>  
> I am trying to find the fastest way to go through a text IFS file and
> replace all occurrences of a string. The file sizes that I am working with
> range from less then 200 bytes to over 80Kb. Obviously the 80Kb one is
> causing the slowness problems that I want to try and remedy.  
Hmm, it seems there is a bug.  If the data you are looking for crosses a
block boundary, your code is not catching it.  For example, assume the
end of a 60000 byte block looks like '&qu' and the beginning of the next
block is 'ot;', then you would not be replacing the &quot with the '"'.

>  
> Below is what I am doing essentially. It only takes two reads from the IFS
> file to get all of the data, but I was wondering if there is any way to do a
> find and replace faster on that large of a string (65535 varying)?
Your current routine scans the buffer six times.  Three of those (half!)
start with the same character.  If there isn't a '&', then some of the
searches could be avoided.

I haven't had to build or locate a very speedy search and replace, but I
would investigate pulling the entire file into memory (assuming it will
always fit inside 16Mb), then write it out, translating by brute force. 

If that's not fast enough, another possibility would be to investigate
the standard C library.  You might be able to whip up a nice search and
replace function using memcpy and memcmp.

"Algorithms in C" by Robert Sedgewick has some interesting string search
routines.

There are probably some MI functions as well.


Regards,
Rich