Fail2Ban system services demand

More of a network-related question than really “internet”, but I am sure you guys will have an idea: I am running a NextCloud Server with Fail2Ban because it’s facing the outside world for me to synch… from :desert_island: :mountain::bullettrain_front:

I am running a little logalizer called Monitorix on that Raspbian machine, too. This logalizer has a report that highlights “system services demand” which only reports Fail2Ban which ticks up every day from 0 to 8 (so approximately one step every 18 minutes) and I wonder what that means.

fail2ban

Does anyone know? I tried googling it, but this led me to believe that this is more of a Monitorix-specific analysis than something broadly considered important. However, I am always intrigued at sawtooth shaped diagrams.

1 Like

I have just installed MonitorX after reading your post and will see what graph I get from my Pi Nextcloud/fail2ban installation.

I wonder if the sawtooth graph is because you have someone attacking/connecting at a rate limited approach. As it is an incremental hits graph?

Graph so far:

1 Like

Well a bored fail2ban should be doing very little… so it would seem that yours gets regular exercise for some reason. Do you have any interesting logs?

1 Like

Have been using that for ages. Lots of good information. Ultimately I want to built my own router and see my total and instantaneous throughput on my network.

2 Likes

Update few hours later:

Also have you tried the https://scan.nextcloud.com/ test?

Excellent - thank you very much for testing this out for me @rah ! :slight_smile:

I am happy to report that I have an A+ rating on the scan.nextcloud.com (rather: they make great and easy to install software…). Hmm. Wonder what it may be since I don’t have any machines running 24/7 that might cause it.

AHAAAAAAAAAA! Reminds me that I have a phone running 24/7 connected to the server. Maybe I reaches out to synch or say hello or something akin every 20 minutes? I will investigate and report back!

1 Like

I’m new to both of these tools --and have just installed them – but isn’t this graph just showing that fail2ban runs periodically on your system? Wouldn’t system demand count increase every time it scans the log for failed login attempts?

EDIT: I see now in serv.pm source code of monitorix that it’s looking for “BAN” action from fail2ban. Can you open that log and see what IP addresses(s) it’s banning at such a regular interval?

Perfectly possible! :slight_smile: BUT - alas - there is news: logs show (thanks, @PHolder !) that fail2ban logs attempts from a Chinese IP (221.12.124.190) in irregular frequency - but consistently. Now looking for a switch to block that IP permanently.

Looks like this in fail2ban.log:

2020-04-21 19:15:21,920 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:15:21
2020-04-21 19:15:23,887 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:15:23
2020-04-21 19:20:19,060 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:20:18
2020-04-21 19:20:20,855 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:20:20
2020-04-21 19:25:16,611 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:25:16
2020-04-21 19:25:18,579 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:25:18
2020-04-21 19:25:19,008 fail2ban.actions        [1025]: NOTICE  [ssh] Ban 221.12.124.190
2020-04-21 19:35:19,829 fail2ban.actions        [1025]: NOTICE  [ssh] Unban 221.12.124.190
2020-04-21 19:39:39,622 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:39:39
2020-04-21 19:39:40,645 fail2ban.filter         [1025]: INFO    [ssh] Found 221.12.124.190 - 2020-04-21 19:39:40

In the NextCloud configuration panel, there is a section configuring fail2ban. I changed the banning policy to “permanently ban” by putting the timer to -1. Also: send me notifications if someone acts up! Let’s see how this helps.

A final note: it helps to look back through the past records. Turns out that the situation started last Saturday.

screenshot

I’d say: MonitoriX and TWiT for the win! :slight_smile:

If you want to ban these ip’s permanently you could use ipset its a fast and simple way to do it, it will make 1 iptable rule then use a flat text file with your banned ip to load, barley uses any memory or cpu.
Just flat out blocks the ip, now i use a blacklist to block multiple targets like china russia and all the known bad apples that a few people maintain for us.
for example my iptable rule is

iptables -I INPUT -m set --match-set ip-blacklist src -j DROP

then i have the ipset file (just a few lines)

create ip-blacklist hash:net family inet hashsize 65536 maxelem 131072
add ip-blacklist 147.91.131.6
add ip-blacklist 62.122.8.8
add ip-blacklist 184.105.139.121
add ip-blacklist 213.141.232.1

Now the script i use to populate it, sorry cant remember where i got it but sure a little searching you can find others, like on github, but its kinda basic bash

#!/bin/bash
IP_TMP=/tmp/ip.tmp
IP_BLACKLIST=/etc/ip-blacklist.conf
IP_BLACKLIST_TMP=/tmp/ip-blacklist.tmp
IP_BLACKLIST_CUSTOM=/etc/ip-blacklist-custom.conf # optional
list=“chinese nigerian russian lacnic exploited-servers”
BLACKLISTS=(
http://www.projecthoneypot.org/list_of_ips.php?t=d&rss=1” # Project Honey Pot Directory of Dictionary Attacker IPs
http://check.torproject.org/cgi-bin/TorBulkExitList.py?ip=1.1.1.1” # TOR Exit Nodes
http://www.maxmind.com/en/anonymous_proxies” # MaxMind GeoIP Anonymous Proxies
https://www.maxmind.com/en/high-risk-ip-sample-list” # MaxMind GeoIP Anonymous Proxies
http://danger.rulez.sk/projects/bruteforceblocker/blist.php” # BruteForceBlocker IP List
http://blocklist.greensnow.co/greensnow.txt” # GreenSnow
#“https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset” # Firehol Level 1
#“https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/stopforumspam_7d.ipset” # Stopforumspam via Firehol

http://danger.rulez.sk/projects/bruteforceblocker/blist.php” # BruteForceBlocker IP List
http://rules.emergingthreats.net/blockrules/rbn-ips.txt” # Emerging Threats - Russian Business Networks List
http://www.spamhaus.org/drop/drop.lasso” # Spamhaus Don’t Route Or Peer List (DROP)
http://cinsscore.com/list/ci-badguys.txt” # C.I. Army Malicious IP List
#“http://www.openbl.org/lists/base.txt” # OpenBL.org 30 day List
https://www.autoshun.org/download/?api_key=97e77a84e7fe645333de16ee0a&format=html” # reg
http://lists.blocklist.de/lists/all.txt” # blocklist.de attackers
)
for i in “${BLACKLISTS[@]}”
do
curl “$i” > $IP_TMP
grep -Po ‘(?:\d{1,3}.){3}\d{1,3}(?:/\d{1,2})?’ $IP_TMP >> $IP_BLACKLIST_TMP
done
for i in echo $list; do
# Download
wget --quiet http://www.wizcrafts.net/$i-iptables-blocklist.html
# Grep out all but ip blocks
cat $i-iptables-blocklist.html | grep -v < | grep -v : | grep -v ; | grep -v # | grep [0-9] > $i.txt
# Consolidate blocks into master list
cat $i.txt >> $IP_BLACKLIST_TMP
done

sort $IP_BLACKLIST_TMP -n | uniq > $IP_BLACKLIST
rm $IP_BLACKLIST_TMP
wc -l $IP_BLACKLIST

ipset flush ip-blacklist
egrep -v “^#|^$” $IP_BLACKLIST | while IFS= read -r ip
do
ipset add ip-blacklist $ip
done

If you look I do have some of the list commented out because they were gone or not working for me.
so it will block tons of ip something like 45000 lines in the file and there are some /16 and /24 networks within it so do a little reading on ipset and i think it will be what your looking for, btw you dont have to use any script to load if you wanted to just add a few to a list its as simple as ipset add ip# yourblacklist name.

right before your for (quoted) I’d throw in a

echo “” >$IP_BLACKLIST_TMP

or the equivalent to make sure no old data was accidentally left from a previous (or previously aborted) run.

yes I see your point and agree, now in my case i run that when i think about it like a few times a year, so there is always a reboot (kernel update) in between. and i did some diff on the results a few years back and really not a lot of changes for me to run it all the time, plus with fail2ban running and have a script to parse that log file of bans to add to my ipset bans i pretty covered anyway dont have ssh open its mostly just for bad web attacks.
Nice catch that never came to mind the var would be holding and can cause a problem.

Can you blacklist it at your firewall? Not letting it in, past your perimeter is always the best course of action.

Also, unless you are planning to travel to those areas (or have friends there you want to share with), I’d look at blocking China completely, along with the IP-blocks for Russia, North Korea, Iran and possibly Israel, although the last one probably isn’t very interested in you as an individual.

In fact, it would probably be easier to just white list the local ISP and your mobile provider(s), possibly your place of work, if you are allowed to access it from there. Whitelisting is always the safer bet, if you can.

I was always a fan of port knocking, in theory, although I’ve never actually tried that for a practical service. For a service that is just supposed to be used by me, I like the idea of blocking everyone who can’t do the port knock.

The issue seems resolved. :slight_smile: Thanks everyone!

Unbenannt1

Great idea and helpful guide, thanks @jeff !

Excellent point, too! I will have to look into that. I tried iptables before but usually screwed something up so that I ended up reverting to the standard perceived safety provided by the router box. However, if I start poking holes into that, its painfully obvious that I should do something to guard the “system behind the poked hole” better. Good point!

Yes, also in conjunction to what @big_D said above - makes sense.

Thank you very much, everyone! :slight_smile:

I used to use pfSense, although I now have a Unifi Security Gateway (I use Unifi access points, switches and the gateway at home at the moment).

If you have a spare PC sitting around, putting it between your modem and your network and plonking pfSense on it is a quick and easy way to protect your network.

Obviously, if your modem/router is also your wireless AP, it isn’t as easy.
My current set-up is:

  • Fritz!Box as modem/router, Wi-Fi disabled
  • DeVolo dLAN 1200 to my office in the cellar
  • Unifi USG as gateway onto the local network
  • Unifi 8 port PoE switch and Zyxel 24 port switch
  • 2 Unifi access points, 1 in wireless bridging / mesh mode

Probably overkill for most people, but I use the VLAN capabilities of the Unifis and the Zyxel to split out the traffic to dedicated networks E.g. management interfaces, general traffic, VOIP traffic, IoT traffic, WLAN and guest WLAN.

Yeah, sounds interesting. To be honest, I am toying with the idea of moving “everything” to the cloud and simply getting rid of the Raspberrys, disks, additional network equipment. Sure, it is fun - but I begin to realise the inefficiency and negative impact on IT security of my limited skill and attention. Plus: less energy wasted and a board in the cabinet won back… Not to go needlessly negative, simply to consider: have the admin stuff done by actual admins. Then again, I have a feeling that I’d just move the stuff to the basement and, six months later, bring it back up again to once again play admin for my five files. :smiley: It is a fun hobby. That’s what golfing must feel like, just sitting down.

Yes, it depends on what you want to do. But “moving” everything to the cloud means you also have to hope that the people running the service you are using are competent - not always a give. :frowning:

And if you move your NextCloud to the cloud or to a dedicated server, you still have to ensure it is correctly configured and administered.

That image is interesting. It looks like you are running NexcloudPi. I do not have those options in the Nextcloud installed from Nexcloud. I installed all manually, Raspbian, LAMP stack, Nextcloud, Fail2ban, Lets Encrypt etc. I can’t remember why I did not go the NextcloudPi route now, there was a reason. I like the Fail2ban and UFW integration showing in your screen grab.

1 Like