How to block Bad Bots (User Agents) using .htaccess
Some bots can cause a high load on servers as they index too many pages or get into some never-ending loop.
last day one of the servers I manage had a very high load. On checking apache logs, i have thousands of access like
|
1 2 3 |
135.181.138.45 - - [24/Aug/2022:03:00:54 +0000] "GET /sv/produkt-linux-server/page/7/?add_to_wishlist=42752&_wpnonce=a8836de6af HTTP/1.1" 200 209072 "https://domain/sv/produkt-linux-server/page/7/?add_to_wishlist=56196&_wpnonce=1e5a94622c" "Mozilla/5.0 (compatible; SeekportBot; +https://bot.seekport.com)" 135.181.138.45 - - [24/Aug/2022:03:00:53 +0000] "GET /sv/produkt-linux-server/page/7/?add_to_wishlist=66324&_wpnonce=a8836de6af HTTP/1.1" 200 209072 "https://domain/sv/produkt-linux-server/page/7/?add_to_wishlist=56196&_wpnonce=1e5a94622c" "Mozilla/5.0 (compatible; SeekportBot; +https://bot.seekport.com)" 135.181.138.45 - - [24/Aug/2022:03:00:54 +0000] "GET /sv/produkt-linux-server/page/7/?add_to_wishlist=42830&_wpnonce=a8836de6af HTTP/1.1" 200 209072 "https://domain/sv/produkt-linux-server/page/7/?add_to_wishlist=56196&_wpnonce=1e5a94622c" "Mozilla/5.0 (compatible; SeekportBot; +https://bot.seekport.com)" |
In this case, the bot user agent is
|
1 |
Mozilla/5.0 (compatible; SeekportBot; +https://bot.seekport.com) |
To block the bot, I added the following code in .htaccess file
|
1 2 3 |
RewriteEngine On RewriteCond %{HTTP_USER_AGENT} (SeekportBot|SpamBot2) [NC] RewriteRule (.*) - [F,L] |
This will block any visitor with Browser User Agents SeekportBot or SpamBot2.
To block common marking bots, run
|
1 2 3 |
RewriteEngine On RewriteCond %{HTTP_USER_AGENT} (semrush|ahref|mj12bot) [NC] RewriteRule (.*) - [F,L] |
If you are using Nginx web server, see How to block bad bots User-Agents in Nginx or using Block User-Agent using Cloudflare
Back to htaccess