Nginx Log Parsing

Identifying malicious requests

Malicious bots are always looking for attack vectors on to websites, these requests can eat up server resources. It is good practice to identify the urls that bots are targeting such as: * Adminer url * Obvious admin urls (/admin, /backend, etc) * Sensitive files or dot files

Identifying urls hit by an IP address

cd /var/log/nginx
grep 'IP Address' *access*.log  | grep -Po '(?<=GET )([^" ]+)' | sort -u

This is useful when you notice that an IP is hitting some malicious endpoints. It will return all of the uris that were hit by this ip.

The output can be used to populate a block list or honey pot to block other bots.

Example output

...
/adminer-3.0.0.php
/adminer-3.0.1.php
/adminer-3.1.0.php
/adminer-3.2.1.php
/adminer-3.2.2.php
/adminer-3.6.1-mysql.php
...