× The internal search function is temporarily non-functional. The current search engine is no longer viable and we are researching alternatives.
As a stop gap measure, we are using Google's custom search engine service.
If you know of an easy to use, open source, search engine ... please contact support@midrange.com.



We make available access to a large database using CGI programs. More and more frequently, I am seeing attempts to download the entire data base of millions of records by hitting the server with a script to sequentially access every record.

I'm trying to come up with a method of preventing this kind of wholesale download. Many web sites now use a challenge system consisting of an authorization word in graphics format that the user must key in before being granted access. The problem is that this solution imposes a burden on our legitimate users that shouldn't be necessary, so I'm looking for a solution that doesn't involve any additional steps for our users.

The first few times this happened, I changed the Apache server configuration to deny access to the offending IP address, but it's impossible to monitor log files to keep up with the ever-changing IP addresses where these attacks originate.

Anyone have any thoughts on how to approach this problem?


As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.