Finding a hole in a sequence using SQL vs native I/O

Wilt, Charles wrote:

You're correct given the way you've coded the SQL.

However, my conclusions' were based on Michael's SQL. While not 100%, I'm pretty sure his SQL won't
need the temp table or read the index more than once behind the scenes.

But testing would tell us for sure.

Testing confirms for me that SQL is indeed faster if the first "hole" is somewhere out past 60-70% of the way into the file. This is consistent with the idea that the more I/O you do, the more likely SQL is to process better.

Interestingly, both SQLs perform about the same way. On my machine, it takes about 7 seconds to find a hole halfway into an 900,000 record file (using either SQL - the difference seems negligible, but I haven't done a bunch of testing). It takes about 5.7 seconds for the read. At 600,000 it takes 8 seconds for the SQL, and about 9.7 for the read. At the end, it takes 9 seconds for the SQL, and nearly 14 for the read.

How would this be faster than just reading 500,000 records
sequentially? There is no magic in SQL, except that it doesn't need to
bounce back and forth between the HLL and the SLIC. But I can't see how
that will help this particular problem.

My conclusions assume Michaels' SQL will read through the index once.

Thus, the SQL would be faster....

What's interesting is the consistency in the SQL approach. It seems to indicate that is a temporary index being built. Even if the 10th record is not found, the SQL still takes around 7 seconds. And as you can guess, with the read technique, it takes milliseconds. My conclusion is that if your holes are interspersed throughout your file, native I/O is your answer. If your holes are consistently at the end of the sequence, go with SQL.

Joe