I had the same situation in the past. My approach is:
- Looking to apache/nginx/haproxy access logs, and then my Perl script will login to any row returned with 4XX, 5XX http code.
- If I can see there are a lot of 4XX, 5XX from one IP within 1-2 mins (let's say more than 10 times). I will update my firewall to deny it (I suggest use firewall to limit instead of http acl (return 403).
- After that, my script will send slack/email to notify to the team.
With my setup, it is flexible and tiny to install anywhere. If you are happy with this, I will help to write a script for you.
Regards,
--T