How To Block Bad Bot or Host (Free PHP API)?

If you are a website owner or server admin then you might have faced the issue of bad bots scraping your site content. Also, these bots hit your sever or site database with an unlimited crawling rate. Usually, scrappers create bots with fake User-Agent and it’s quite hard to check the log for each IP address. There are a few hosts (like Amazonaws, Digitalocean, etc) which are the favorite choice of these scrappers as they are cheap and easily deployable. It’s not much easy to block these hosts as they have a large number of IPV4 or IPV6 ranges. To overcome this issue, I have created a free PHP API to block bad bots or hosts without any extra effort.

What is an API?

API stands for Application Programming Interface, which is a software intermediary that allows two applications to communicate or share/access data. Most of the major websites like Youtube/Facebook allow their user to fetch data from their servers through Application Programming Interface.

Is this system really works and free?

This API works completely free without any limits and is applicable for both IPV4 and IPV6 addresses. It is a bulletproof system to block any host or network operator in this world. This API is developed by me and I am using it for last 3 years and it is working perfectly for me. This API is blazing fast and even you can cache the response if you need to log the details of fake bot IP address. This system works in two-part. First one is to detect the host of IP address and second is to block the host. This system can handle blocking each of the mentioned hosts. For instance, if a host is using million of IP’s then this system can block every IP of that host at every request and even you can save the response for further action.

How to find the Host of Bad Bot?

This API is written in PHP but can be used in all other programming languages. This API returns JSON response which can be used at any other platform. It’s quite fast and easy to deploy. Here are the steps below:

First, find the IP address of bad bots. You can easily check it from server access logs. If you are unable to access the logs, then let me know in the comment box. I will share another code to track the IP’s hitting your site/server along with numbers of connections. Once you found the IP addresses, change the IP to your found IP and you can find the hostname at given URL.

https://livtutor.com/wp-api/host.php?ip=0.0.0.0

Now, we have found the host of IP Address. We just need the first name of domain nameserver. For instance, IANA from above screenshot.

Now we have the hostname, it’s easy for us to block the complete host whether this host is using multiple IPV4 or IPV6 ranges. Instead of IPV4, you can also use any IPV6 address (like 2001:0000:3238:DFE1:0063:0000:0000:FEFB) to find the host. Now, we have the hostname, we are ready for the next step.

PHP Code to Block Bad Bot or Host

This API returns JSON response, so we need to decode it at our server end. Since I am sharing the PHP code to block bad bot or host but anyone can use it with the language of his/her choice. Create a PHP file and write this code inside it. First of all, we need to find the actual IP address of the upcoming request.

<?php
function getRealIpAddr() {
	if (!empty($_SERVER['HTTP_CLIENT_IP'])) {
		$ip=$_SERVER['HTTP_CLIENT_IP'];
	}
	elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR'])) {
		$ip=$_SERVER['HTTP_X_FORWARDED_FOR'];
	}
	else {
		$ip=$_SERVER['REMOTE_ADDR'];
	}
	return $ip;
}
$ip = getRealIpAddr();
?>

Now, we need to call the API to verify whether this IP belongs to blacklisted hosts or not.

<?php
$ip_host = json_decode(file_get_contents('https://livtutor.com/wp-api/block.php?ip='.$ip.'&host=YOUR HOST NAME'), true);
if($ip_host['status'] == 'true') {
		die('You are blacklisted');
		exit;
	}
?>

You can include the above code on the page where bad bots are scraping your content. This API is capable enough to block multiple hosts. If you want to block multiple hosts then you can use the below format to block multiple hosts.

<?php
$ip_host = json_decode(file_get_contents('https://livtutor.com/wp-api/block.php?ip='.$ip.'&host=host1,host2,host3'), true);
?>

If provided IP belongs to the blacklisted host then you will get the status response to true and also you can modify the response to that IP.

This API is completely free and can be used at any platform like WordPress or any other site. This API can be easily used with other programming languages like JAVA, JavaScript, Python or C etc. We recommend you to cache the response to avoid any response delay or preventing the excessive use of server resources at API end. Feel free to suggest or request in the comment box below.