Fighting the pen-test with regex and sweat

Regex. I think its close to becoming sentient. My first experience with it was in the glorious days of Usenet and c-news (no b-news for me), and a chap by the name of Henry Spencer who wrote it about the time I started University. I can still recall m first thread (it was on uw.general about squirrels getting into the Davis Centre. People were passionately pro/con squirrel!) sitting in MC using a DEC VT340 to a BSD 4.x VAX. Today regex is the ‘swiss army chainsaw’ that powers, well, nearly all security devices. Its what makes awk sing, what makes python py.

Tonight my Home Assistant chirped a lot more than usual about log usage, and, fail2ban. So I dug in. But I found most of the culprits were not getting the temporary ban, they were going on about their business.

You see, my default site does a 301 redirect from /. So its unusual for something to get a lot of 301’s (e..g trying /foo, /bar, etc). But, they do indeed go in ‘access_log’ not ‘error_log’ since they are technically not errors.

Well, no more. I added a new filter (nginx-301) and told it to watch access_log. It looks for 301 redirects. If it keeps seeing them from a host, well, it goes in the timeout box for a few minutes. The net effect is, well, either know the URL you are going to, or obey the redirect you get the first time or two. If you hunt for things that don’t exist, I’ll slow you down so much you will find nothing.

# cat nginx-301.conf 
# Fail2Ban filter to match web requests for selected URLs that don't exist
# on the default site

[INCLUDES]

[Definition]

failregex = ^ \- \S+ \[\] \"(GET|POST|HEAD) \/\S+ \S+\" 301 .+$

ignoreregex = 

datepattern = {^LN-BEG}%%ExY(?P<_sep>[-/.])%%m(?P=_sep)%%d[T ]%%H:%%M:%%S(?:[.,]%%f)?(?:\s*%%z)?
              ^[^\[]*\[({DATE})
              {^LN-BEG}