https://perishablepress.com/7g-firewall-log-blocked-requests/
This tutorial explains how to log requests that are blocked by the 7G Firewall. This is useful for testing, debugging, and just keeping an eye on things. Learn how to log requests from Apache mod_rewrite and download my custom 7G logging script. It’s a complete example that shows how to log rewrite requests via PHP. All open source and free 🙂
Contents
- How it works
- Log Example
- Usage
- Configure 7G for logging
- Preparing the Log file
- Reading the log file
- Preflight Check
- Download 7G Logging script
- Troubleshooting 7G Firewall
- Bonus: IP Logging with 7G
How it works
The 7G Firewall includes built-in logging directives. Once logging is enabled (as explained below), mod_rewrite will send request data to our handy little PHP script. The logging script then parses the data and writes it to the log file. The result is a log file that looks very similar to Apache defaults.
Under the hood, here is what happens for any requests blocked by 7G:
- 7G checks the request
- If not blocked, the request continues normally
- If blocked, the request data is sent to the logging script
- The logging script then writes the data to the log file
- The script exits with a simple message (configurable)
So regular visitors and legit traffic will continue normally, while any blocked requests are redirected to the logging script. Conceptually simple.
Log Example
Here is an example showing what the log entries look like:
012.012.012.01 - 2019/01/24 01:47:12 - GET - HTTP/1.0 - /www.php - boot.ini [boot.ini] - - - Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6)
187.255.215.91 - 2019/01/24 21:19:01 - GET - HTTP/1.0 - / - - - - Mozilla/5.0 (compatible; Exabot/3.0; +http://www.exabot.com/go/robot) [Exabot]
222.333.444.00 - 2019/01/24 00:15:33 - POST - HTTP/1.0 - /xertive.php [xertive] - - - - curl/7.54.0
In log entries, matching firewall patterns are indicated via brackets like [this]
. In the above example, notice the following blocked patterns:
boot.ini
— from 7G QUERY STRING rulesExabot
— from 7G USER AGENT rulesxertive
— from 7G REQUEST URI rules
So for each request, the location of the bracketed/match string tells us the related section of 7G. More details on this later in the post.
Usage
To log any/all requests blocked by 7G, follow the steps:
- Add 7G Firewall to root .htaccess (or Apache config)
- Configure 7G Firewall for logging (see next section)
- Unzip 7G Logging script and upload files to root web directory
- Make
log.txt
writable (and protect via .htaccess)
Test well and leave feedback below in the comments or direct via my contact form. Please do not report bugs in the comment section, best to communicate via email/contact form, thank you for understanding.
Configure 7G for logging
The 7G Firewall comprises the following sections:
# 7G:[CORE]
# 7G:[QUERY STRING]
# 7G:[REQUEST URI]
# 7G:[USER AGENT]
# 7G:[REMOTE HOST]
# 7G:[HTTP REFERRER]
# 7G:[REQUEST METHOD]
Each of these sections contains a set of mod_rewrite rules. For each section (except CORE rules), you want to disable (comment out) the default RewriteRule, and enable (un-comment) the logging RewriteRule. For example, in the QUERY STRING section:
# 7G:[QUERY STRING]
<IfModule mod_rewrite.c>
.
.
.
RewriteRule . - [F,L]
# RewriteRule . /7g_log.php?log [L,NE,E=7G_QUERY_STRING:%1___%2___%3]
</IfModule>
We want to change that to this:
# 7G:[QUERY STRING]
<IfModule mod_rewrite.c>
.
.
.
# RewriteRule . - [F,L]
RewriteRule . /7g_log.php?log [L,NE,E=7G_QUERY_STRING:%1___%2___%3]
</IfModule>
So now the QUERY STRING rules are sending request data to our 7g_log.php
log file. Repeat this same “rule swap” for each section (except CORE rules) in 7G. Once this is done, 7G is configured for logging and will send all blocked requests and data to the 7G Logging script.
Preparing the Log file
To prepare the log file, you want to make sure it is writable by the server (e.g., change CHMOD permissions on the file). Then you also want to make sure the file is protected from outside access. This is prevent sensitive information from falling into public domain. As we’re already working with .htaccess, here is a simple code snippet that protects our log file from all outside access:
# Apache < 2.4
<IfModule !mod_authz_core.c>
<Files ~ "7g_log.txt">
Deny from all
</Files>
</IfModule>
# Apache 2.4+
<IfModule mod_authz_core.c>
<Files ~ "7g_log.txt">
Require all denied
</Files>
</IfModule>
Add that to your root .htaccess
file and done. No changes need to be made. Note however that both sections are not necessary; if you know your version of Apache, it is safe to use only the relevant code.
Reading the log file
For each log entry, the following request data is recorded (if available):
- IP Address
- Date/Time
- Request Method
- Request Protocol
- Request URI
- Query String
- Remote Host
- Referrer
- User Agent
So each log entry/line records these fields in the following order:
IP Address - Date/Time - Request Method - Request Protocol - Request URI - Query String - Remote Host - Referrer - User Agent
Note that empty/blank values are simply left as-is. So when you see stuff like this in your log file it’s totally normal:
- / - - - - User agent 1234..
It just means that no data was available for each of the blank items. It is common especially for Remote Host and Referrer fields to be blank.
Preflight Check
Here is a checklist of requirements for logging to work:
- 7G Firewall installed and configured for logging
7G_log.php
and7G_log.txt
both added to root web directory7G_log.txt
is writable by the server7G_log.txt
is protected against outside access
Once everything is in place and ready, 7G logging should be enabled on your site. Note that the 7G_log.php
script includes several variables/options that you can modify if desired:
SEVENGLOGPATH
= Path to the log fileSEVENGLOGFILE
= Name of the log fileSEVENGUALENGTH
= Length of User Agent log entrySEVENGEXIT
= Message displayed to blocked visitorsdate_default_timezone_set('UTC')
= Default Timezone
These variables are predefined to work according to this tutorial, out of the box. So no changes need to be made to the logging script, but there are few things that you can tweak as needed.
Download 7G Logging script
Here is the latest version of the 7G Firewall Logging script. License: GPL v3.
Troubleshooting 7G Firewall
Once logging is enabled on your site, troubleshooting and testing the 7G Firewall becomes quick and easy. This section gives some ideas and tips that should be useful for anyone wanting to debug, test, and so forth.
While using 7G, if any legitimate page or resource fails to load, or if some feature stops working, simply take a look at the 7G log file. Locate the most recent (and/or relevant) entries. For each related entry, the offending firewall pattern will be indicated with [brackets]
. So you will know immediately which 7G rules/pattern(s) are responsible for any issues.
If you discover any offending patterns, simply disable (comment out) or remove. Also please report any bugs or patterns that are causing problems. That way I can update the firewall and keep things as error-free as possible.
Troubleshooting Tips
Here is a working list of things to check when troubleshooting 7G.
- Locating patterns
- Some sections of 7G contain a lot of “tightly packed” rules. This can make locating offending patterns difficult, even when you know the matching pattern. In such cases, the quickest way to identify the pattern is to use the the halving method.
- Code placement
- In your
.htaccess
file, Make sure that 7G code before any existingmod_rewrite
rules (e.g., WordPress Permalinks). - Server error
- If you get a server error after installing 7G, double-check that your site meets the requirements.
- Encoded Characters
- By default, special characters are passed to the log file unencoded. To instead pass the encoded equivalents, remove the “No Escape” flag
NE
from each of the 7G rewrite rules.
More tips will be added as they arrive.
Bonus: IP Logging with 7G
Users familiar with nG Firewall/Blacklist may have noticed that IP blocking is removed from 7G. As covered in previous posts, there are better, more effective ways to prevent site access based on IP address. Nonetheless, blocking by IP remains useful in a variety of scenarios. So with that in mind, here is a bonus IP ADDRESS section that may be added to 7G (or any .htaccess file):
# 7G:[IP ADDRESS]
<IfModule mod_rewrite.c>
# RewriteCond %{REQUEST_URI} !(7g_log.php) [NC]
# RewriteCond %{REMOTE_ADDR} ^(000.000.000.000)$ [OR]
# RewriteCond %{REMOTE_ADDR} ^(100.100.100.100)$ [OR]
# RewriteCond %{REMOTE_ADDR} ^(200.200.200.200)$
# RewriteRule . /7g_log.php?log [L,NE,E=7G_IP_ADDRESS:%1]
</IfModule>
As written, this code does nothing, because each directive is disabled (commented out with a hash/pound sign #
). To enable these rules, remove the hash from each line. Then replace the “dummy” placeholder IP values with any that you would like to block and log. Once implemented, these rules will result in any blocked IP addresses indicated in the log file with [brackets]
.
RewriteCond
should end with an [OR]
flag, except the last RewriteCond
. If you only want to block one IP address, use only one RewriteCond
and omit the [OR]
flag.May the log files be with you!
Source: Security Feed