I’m hosting some web services and I’ve been using fail2ban (http://www.fail2ban.org) on my servers to block offenders and report bad IP’s thanks to AbuseIPdB (https://www.abuseipdb.com/).

I’ve been using also the Suricata package (https://suricata-ids.org/) on my PfSense appliance (https://www.pfsense.org/) to block more frequent upstream offenders (Scans…) but without sharing this noisier data.

But the temptation is just too strong, I want this without adding anything to the stuff available on BSD :

No Python…
No Lua…
No Bourne Again Shell (Bash)…
Just the Bourne Shell (sh) ~~or php~~.

I’m not used to sh nor to vi, but ok let’s get rid of fancy windows, nice colors and high level functions… let’s go coding something.

What is the data to process ?

As expected Suricata’s log file is quite detailed, and located on:

/var/log/suricata/INTERFACE/block.log for me it looks like:

/var/log/suricata/suricata_igb07936/block.log

Where is data sent ?

Thanks to AbuseIddB API I’ll report everything thanks to the REPORT EndPoint function (https://docs.abuseipdb.com/#report-endpoint):

# POST the submission.
curl https://api.abuseipdb.com/api/v2/report \
–data-urlencode “ip=127.0.0.1” \
-d categories=18,22 \
–data-urlencode “comment=SSH login attempts with user root.” \
-H “Key: $YOUR_API_KEY” \
-H “Accept: application/json”

What shall the script do ?

Read the Block.log
Parse each NEW line
Process each line
1. If the line seems not to be reported: PASS the line
  1. if the IP seems to be mine 🙂
  2. if suricata rule’s “msg” is not to report (for example I don’t want to report “ET CINS Active Threat Intelligence Poor Reputation IP”…
2. Otherwise: REPORT the line
When finished store somewhere the current count of lines

The script I came to…

So here is the script

#!/bin/sh

log_dir=”/var/log/suricata/suricata_igb07936″ #don’t forget to set the right gb#

process_line()
{
echo “”
echo “$1”
echo “”

Attack=$(echo $1 | sed ‘s/.* \[Block .*\] \[\*\*\] \[[0-9]*:[0-9]*:[0-9]*\] $.*$ \[\*\*\] \[.*\] {.*} $.*$:[0-9]*$/\1/’)
Ip=$(echo $1 | sed ‘s/.* \[Block .*\] \[\*\*\] \[[0-9]*:[0-9]*:[0-9]*\] $.*$ \[\*\*\] \[.*\] {.*} $.*$:[0-9]*$/\2/’)

if echo “$Attack” | grep -q “CINS Active Threat Intelligence Poor Reputation IP”; then
echo “Passing $Ip $Category $Attack”
elif echo “$Attack” | grep -q “ET COMPROMISED”; then
echo “Passing $Ip $Category $Attack”
elif echo “$Attack” | grep -q “ET DROP Dshield Block Listed Source”; then
echo “Passing $Ip $Category $Attack”
elif echo “$Attack” | grep -q “ET DNS Query for .to TLD”; then
echo “Passing $Ip $Category $Attack”
elif echo “$Attack” | grep -q “ET TOR EXIT NODE”; then
echo “Passing $Ip $Category $Attack”
elif echo “$Attack” | grep -q “SCAN”; then
echo “report “$Ip” “14” “$Attack”” # 14 = PORT SCAN
report “$Ip” “14” “$Attack”
elif echo “$Attack” | grep -q “NMAP”; then
echo “report “$Ip” “14” “$Attack”” # 14 = PORT SCAN
report “$Ip” “14” “$Attack”
else
echo “report “$Ip” “15” “$Attack”” #15 = Hacking
report “$Ip” “15” “$Attack”
fi

echo “”
echo “——“
}

report()
{
YOUR_API_KEY=XXXXX #Don’t forget to set your API_KEY

if [ $# -ne 3 ]
then
echo “This script require 3 arguments IP Category Message”
else
#echo “>>>>reporting “ip=$1” categories=$2 –data-urlencode “$3″”

curl https://api.abuseipdb.com/api/v2/report –data-urlencode “ip=$1” -d categories=$2 –data-urlencode “comment=$3” -H “Key: $YOUR_API_KEY” -H “Accept: application/json”
fi
}

echo “############################################################################”
echo `date`
echo “”
echo “C’est parti…”
echo “”

#fichiers=$(ls $log_dir | grep –color “block\.log\.[0-9_]\{1,\}”)
fichiers=$(ls $log_dir/block.log)
echo “$fichiers”

for fichier in $fichiers
do
compteur=1 # Ligne n°1
prev_linecount=`cat “$fichier.lwc”`

if [ $? -ne 0 ] ;
then
echo “Je n’arrive pas a lancer la commande cat $fichier.lwc: Par defaut c’est 0”
prev_linecount=0
fi

echo “Nombre de lignes précédemment dans le fichier $fichier: $prev_linecount”

current_linecount=`wc -l “$fichier” | awk ‘{ print $1 }’`

echo “Nombre de lignes actuellement dans le fichier $fichier: $current_linecount”

if [ $(($current_linecount – $prev_linecount)) -lt 0 ] ; then
prev_linecount=0
fi

toread_linecount=$(($current_linecount – $prev_linecount))

echo “Nombre de lignes à importer: $toread_linecount”

echo “”
echo “——“
tail -n $toread_linecount $fichier |
{ while IFS= read -r line
do
echo $compteur “/” $toread_linecount
process_line “$line”
compteur=$(($compteur+1))

#sleep 0.5
done
}

echo $current_linecount > “$fichier.lwc”

done

exit 0

Installation on PfSense

Thanks to CRON tasks I schedule:

*/20	*	*	*	*	root	/usr/bin/nice -n20 /PATH TO YOUR SCRIPT/abuseipdbreport.sh >> /var/log/abuseipdb/report.log

Conclusion

This works nicely:

The log is populating as expected:

############################################################################
Tue Jun 15 22:00:00 CEST 2021

C’est parti…

/var/log/suricata/suricata_igb07936/block.log
Nombre de lignes précédemment dans le fichier /var/log/suricata/suricata_igb07936/block.log: 262
Nombre de lignes actuellement dans le fichier /var/log/suricata/suricata_igb07936/block.log: 274
Nombre de lignes à importer: 12

——
1 / 12

06/15/2021-21:42:50.684801 [Block Src] [**] [1:2402000:5941] ET DROP Dshield Block Listed Source group 1 [**] [Classification: Misc Attack] [Priority: 2] {UDP} 192.241.214.180:60409

Passing 192.241.214.180 ET DROP Dshield Block Listed Source group 1

——
3 / 12

06/15/2021-21:43:58.710003 [Block Src] [**] [1:2009582:3] ET SCAN NMAP -sS window 1024 [**] [Classification: Attempted Information Leak] [Priority: 2] {TCP} 114.104.188.248:53306

report 114.104.188.248 14 ET SCAN NMAP -sS window 1024
{“data”:{“ipAddress”:”114.104.188.248″,”abuseConfidenceScore”:100}}

——
12 / 12

06/15/2021-21:58:24.919030 [Block Src] [**] [1:2500118:5830] ET COMPROMISED Known Compromised or Hostile Host Traffic group 60 [**] [Classification: Misc Attack] [Priority: 2] {TCP} 209.141.59.244:58975

Passing 209.141.59.244 ET COMPROMISED Known Compromised or Hostile Host Traffic group 60

——

And reports are shown in AbuseIPDb API :

[Show slideshow]