Broken LAMPS; Apache (Part 2)

by Posted on

As spoken prior, “Spring has sprung, and all about us, the frantic rush to clean up from all the things ignored over the long winter of March 2020 is in progress. Many LAMPS and WordPress sites are awaking to find the latest set of requirements, guidelines, and best practices from the OWASP and W3C groups leaves them in a questionable state.”

In this missive, we will review basic apache installation and lockdown. Again CIS has benchmarks for the latest version, and they are worth the time to peruse and also again, the application of these should match your standards.

I will once again caution the readers of this missive of the following:

  1. DO NOT TEST THIS IN PRODUCTION.
  2. DO NOT APPLY THINGS YOU DO NOT UNDERSTAND.
  3. MAKE SURE YOU HAVE TESTED BACKUPS OF THE SERVER BEFORE YOU BEGIN.
  4. MAKE SURE YOU BACKUP FILES BEFORE YOU EDIT THEM.
  5. MAKE SURE YOU TEST THE FUNCTIONALITY OF YOUR SITE BEFORE YOU BEGIN.
  6. BREAK ONE THING AT A TIME.
  7. TEST EACH CHANGE.
  8. KEEP NOTES ON WHAT WAS CHANGED, AND WHEN.

We begin with the installation of apache from a known good repository. Along with this, we shall also install the mod_security, mod_evasive, and mod-headers libraries. For this purpose, we shall use the host test.techhell.org; it has an elastic IP assigned and is open to 0/0 on ports 80 / 443. (For those reading this, It is a test instance in an isolated VPC and contains no data.

Mod_security is a web application firewall (WAF). When the mod_security2 module is added to a server, we should install it with the OWASP Core Rule Set (CRS). OWASP CRS is a list of rules created for WAFs and protects against various common attacks, like SQL injection, cross-site scripting, and local file inclusion. You can find a full list of common attacks that the OWASP Core Rule Set protects in this OWASP CRS article. This in and of itself is deserving of a missive, so I shall defer further discussion of this into that (future) missive.

Mod_evasive is an Apache web services module that helps your server stay running in the event of an attack. A common type of cyber attack comes in the form of a Denial of Service (DoS), Distributed Denial of Service (DDoS), or brute-force attempting to overwhelm your security.

Mod_headers is an Apache module that allows you to control and modify HTTP request and response headers in Apache. This will allow us to match upcoming OWASP, W3C, and chrome guidelines.

We begin with a basic apache install, (after a package repository refresh):

# Refresh repo's
#Systems Updates
sudo apt-get update
sudo apt-get upgrade -y
sudo apt-get autoremove
sudo apt-get autoclean

# Install apache2
sudo apt install apache2

After completion a browser visit to http://test.techhell.org yields the default apache web page.

We shall conduct a basic security scan of this site using the Mozilla Observatory scanner for demonstration purposes. As expected, we fair, “POORLY.”

From the above line items (HTTP Strict Transport Security and Redirection), we can note that the BASIC requirement of HTTPS is not present. We can correct this by applying an SSL certificate; the simplest way to do so is via CertBot. The application of said is documented there and in many places within the net. In its simplest form, installation is as follows:

#Install the CertBot Bot
sudo snap install --classic certbot

# Request and install our cert
ubuntu@test-4-164:~$ sudo certbot --apache
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator apache, Installer apache
Enter email address (used for urgent renewal and security notices)
 (Enter 'c' to cancel): crawls@techhellx.org

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Please read the Terms of Service at
https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf. You must
agree in order to register with the ACME server. Do you agree?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: Y

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Would you be willing, once your first certificate is successfully issued, to
share your email address with the Electronic Frontier Foundation, a founding
partner of the Let's Encrypt project and the non-profit organization that
develops Certbot? We'd like to send you email about our work encrypting the web,
EFF news, campaigns, and ways to support digital freedom.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: N
Account registered.
No names were found in your configuration files. Please enter in your domain
name(s) (comma and/or space separated)  (Enter 'c' to cancel): test.techhell.org
Requesting a certificate for test.techhell.org
Performing the following challenges:
http-01 challenge for test.techhell.org
Enabled Apache rewrite module
Waiting for verification...
Cleaning up challenges
Created an SSL vhost at /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache socache_shmcb module
Enabled Apache ssl module
Deploying Certificate to VirtualHost /etc/apache2/sites-available/000-default-le-ssl.conf
Enabling available site: /etc/apache2/sites-available/000-default-le-ssl.conf
Enabled Apache rewrite module
Redirecting vhost in /etc/apache2/sites-enabled/000-default.conf to ssl vhost in /etc/apache2/sites-available/000-default-le-ssl.conf

As you can see, the cert has been installed, and our basic apache configuration has been modified to use said. A visit to http://test.techhell.org redirects us to https://test.techhell.org, and we have a valid certificate.

And we will rerun our basic security scan. To find we have passed one additional test, but are still failing.

We note that a number of the failing tests are header-based. To correct these, we shall need to enable the mod_headers module and configure it to include/modify the HTTP response headers. (Using a standard apache2 installation mod_headers is already installed)

# Enable mod_headers
ubuntu@test-4-164:~$  sudo a2enmod headers
Enabling module headers.
To activate the new configuration, you need to run:
sudo systemctl restart apache2
# and test the module is loaded
sudo apachectl -M | grep headers
 headers_module (shared)

As we see above mod_headers is installed and enabled, we now need to configure the response headers.

Let us begin with the HTTP Strict Transport Security configuration. We will edit the virtual host SSL configuration file and add the necessary verses to the VirtualHost stanza. In this case /etc/apache2/sites-enabled/000-default-le-SSL.conf.

HTTP Strict Transport Security (HSTS) is an HTTP header that notifies user agents to only connect to a given site over HTTPS, even if the scheme chosen was HTTP. Browsers that have had HSTS set for a given site will transparently upgrade all requests to HTTPS. HSTS also tells the browser to treat TLS and certificate-related errors more strictly by disabling the ability for users to bypass the error page.

The header consists of one mandatory parameter (max-age) and two optional parameters (includeSubDomains and preload), separated by semicolons.

sudo nano /etc/apache2/sites-enabled/000-default-le-ssl.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>

        ServerAdmin crawls@techhell.org
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

#
# Header Directives
#
## Begin HTTP Strict Transport Security
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"
## End HTTP Strict Transport Security
ServerName test.techhell.org
SSLCertificateFile /etc/letsencrypt/live/test.techhell.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/test.techhell.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>
# After testing we will test our config changes with the command
sudo apachectl configtest
# If our syntax is good we will restart
sudo systemctl restart apache2

A scan after this modification shows an “improvement”, not an absolute fail, but not up to standards.

We will proceed to add the X-Content-Type-Options, X-Frame-Options, X-XSS-Protection headers before additional scans.

X-Content-Type-Options is a header supported by Internet Explorer, Chrome, and Firefox 50+ that tells it not to load scripts and stylesheets unless the server indicates the correct MIME type. These browsers can incorrectly detect files like scripts and stylesheets without this header, leading to XSS attacks. As such, all sites must set the X-Content-Type-Options header and the appropriate MIME types for files that they serve.

sudo nano /etc/apache2/sites-enabled/000-default-le-ssl.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>

        ServerAdmin crawls@techhellx.org
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

#
# Header Directives
#
## Begin HTTP Strict Transport Security
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"
## End HTTP Strict Transport Security
## Begin X-Content-Type-Options
Header set X-Content-Type-Options nosniff
## End X-Content-Type-Options
ServerName test.techhell.org
SSLCertificateFile /etc/letsencrypt/live/test.techhell.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/test.techhell.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

X-Frame-Options is an HTTP header that allows sites control over how your site may be framed within an iframe. Clickjacking is a practical attack that allows malicious sites to trick users into clicking links on your site even though they may appear to not be on your site at all. As such, the use of the X-Frame-Options header is mandatory for all new websites, and all existing websites are expected to add support for X-Frame-Options as soon as possible.

sudo nano /etc/apache2/sites-enabled/000-default-le-ssl.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>
        ServerAdmin crawls@techhellx.org
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
#
# Begin Header Directives
#
## Begin HTTP Strict Transport Security
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"
## End HTTP Strict Transport Security
## Begin X-Content-Type-Options
Header set X-Content-Type-Options nosniff
## End X-Content-Type-Options
## Begin X-Frame-Options
Header always append X-Frame-Options SAMEORIGIN
## End X-Frame-Options
#
# End Header Directives
#
ServerName test.techhell.org
SSLCertificateFile /etc/letsencrypt/live/test.techhell.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/test.techhell.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

X-XSS-Protection It is a feature of Internet Explorer and Chrome that stops pages from loading when they detect reflected cross-site scripting (XSS) attacks. Although these protections are largely unnecessary in modern browsers when sites implement a strong Content Security Policy that disables the use of inline JavaScript ('unsafe-inline'), they can still provide protections for users of older web browsers that don’t yet support CSP.

sudo nano /etc/apache2/sites-enabled/000-default-le-ssl.conf
<IfModule mod_ssl.c>
<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined
#
# Begin
Header Directives
#
## Begin HTTP Strict Transport Security
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"
## End HTTP Strict Transport Security
## Begin X-Content-Type-Options
Header set X-Content-Type-Options nosniff
## End X-Content-Type-Options
## Begin X-Frame-Options
Header always append X-Frame-Options SAMEORIGIN
## End X-Frame-Options
## Begin X-XSS-Protection
Header set X-XSS-Protection "1; mode=block"
## End X-XSS-Protection
#
# End Header Directives
#
ServerName test.techhell.org
SSLCertificateFile /etc/letsencrypt/live/test.techhell.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/test.techhell.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

With each iteration, we shall test our config, restart our apache service and check our applications so that error will not compound, leaving us in an indeterminate state for our troubleshooting. Scanning our site/headers leaves us as follows:

Upcoming / Proposed / Annoying headers on the horizon are the Referrer-Policy Header and the Permissions-Policy Header. We shall touch these, LIGHTLY, here.

The Referrer-Policy controls what information is presented to a webserver whence a user visits. For example, When a user clicks a link on one site, the origin that takes them to another site, the destination, the destination site receives information about the origin the user came from.

We implement this in the /etc/apache2/sites-available/000-default-le-ssl.conf with the directive Header set Referrer-Policy “strict-origin,” which will always set the referrer header to the origin from which the request was made. This will strip any path information from the referrer information.

I will also add a Permissions Policy, a new header that allows a site to control which the browser can use APIs or features.

We implement this in the /etc/apache2/sites-available/000-default-le-ssl.conf with the directive: Header always set Permissions-Policy (POLICY_STRING).. Note the policy as given is restricts all site access to the listed APIs.

My config file with both is as below: The discussed items are in their respective verses.

  GNU nano 4.8                                                       000-default-le-ssl.conf                                                       Modified  <IfModule mod_ssl.c>
<VirtualHost *:443>
  
        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

#
# Begin Header Directives
#
## Begin HTTP Strict Transport Security
Header always set Strict-Transport-Security "max-age=63072000; includeSubDomains"
## End HTTP Strict Transport Security
## Begin X-Content-Type-Options
Header set X-Content-Type-Options nosniff
## End X-Content-Type-Options
## Begin X-Frame-Options
Header always append X-Frame-Options SAMEORIGIN
## End X-Frame-Options
## Begin X-XSS-Protection
Header set X-XSS-Protection "1; mode=block"
## End X-XSS-Protection
## Begin Referrer Policy
Header always set Referrer-Policy "strict-origin"
## End Referrer Policy
## Begin Permissions Policy
Header always set Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()"
## End Permissions Policy
#
# End Header Directives
#
ServerName test.techhell.org
SSLCertificateFile /etc/letsencrypt/live/test.techhell.org/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/test.techhell.org/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

One can perform a security header scan at https://securityheaders.com/. Our scan reports as follows:

For now, we shall leave the elephant in the room (Content Security Policy) alone and start to focus on some more basic items. CSP has started to overtake many other headers in importance and complexity. Most examples of a CSP that allow a web site to function include something similar to:

 default-src 'none'; img-src 'self'; script-src 'unsafe-inline'; style-src 'unsafe-inline'

Note the inclusion of the ‘unsafe-inline parameters disables much of the security policy, which relegates its functionality to a box-checking service. This in and of itself is deserving of a missive, so I shall defer further discussion of this into that (future) missive.

We should NOT advertise the specifics of our web server. AKA ServerSignature and Server Tokens.

sudo nano /etc/apache2/apache2.conf
# Add/modify the following lines to hide server information in Apache.
ServerSignature Off
ServerTokens Prod

Save, test your config and restart the apache service.

Whilst we are bashing about in the apache configuration files, it may be time to restrict a few items. We will do more later as this series progresses, but for now, a couple of basics. First of all, we will be working in the <Directory /var/www/html> stanza (this may need to be created) and will modify this as follows:

<Directory /var/www/html>
        DirectoryIndex index.php index.html
        Options -Indexes +FollowSymLinks
        LimitRequestBody 995367
        AllowOverride All
        Require all granted
</Directory>

The Options -Indexes will prevent listing the directory if an index is not present. The LimitRequestBody will limit the overall size of the request sent to the end client; this may be useful in avoiding some denial of service attacks. The AllowOverride allows your .htaccess file(s) to override the global settings in /etc/apache2/apache2.conf. Note .htaccess files are specific to their location in the file system. I.E. /var/www/.htaccess can be overridden by /etc/www/html/.htaccess, and so on.

And as always we shall test our config and restart apache.

One final item to be touched is the 301, Permanent Redirect directive in /etc/apache2/000-default.conf:

nano 000-default.conf
<VirtualHost *:80>
        #ServerName www.example.com

        ServerAdmin webmaster@localhost
        DocumentRoot /var/www/html
        Redirect permanent / https://<INSERT_YOUR_HTTPS_URL_HERE>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

We spoke prior about installation of mod_evasive. I will install, configure and test said, but as I will be loading other WAF’s it will be of small use and may in fact complicate later operations. One can accomplish this as follows:

# Install mod_evasive
sudo apt-get install libapache2-mod-evasive
# Configure BASICs as follows:
sudo nano /etc/apache2/mods-enabled/evasive.conf
<IfModule mod_evasive20.c>
    DOSHashTableSize    3097
    DOSPageCount        2
    DOSSiteCount        50
    DOSPageInterval     1
    DOSSiteInterval     1
    DOSBlockingPeriod   10

    DOSEmailNotify      crawls@techhellx.org
    #DOSSystemCommand    "su - someuser -c '/sbin/... %s ...'"
    DOSLogDir           "/var/log/apache2/mod_evasive"
    #DOSWhitelist 192.168.1.*
</IfModule>

We shall leave an email notification, and we will leave the DOSSystemCommand commented out for the moment. DO note, cutting and pasting these as given will cause email alerts to be blackholed, so DO configure YOUR email address. We can exclude IP addresses from an evaluation using the DOSWhitelist verse.

A brief explanation of each option is shown below:

  1. DOSHashTableSize: mod_evasive uses this option to control the hash table size. It is recommended to increase this if you have a busy web server.
  2. DOSPageCount: This option specifies the threshold limit for the number of requests allowed to the same URI per second. Once the threshold limit has been exceeded, the client’s IP address will be blacklisted.
  3. DOSSiteCount: This option specifies the limit on the total number of requests allowed to the same IP address.
  4. DOSPageInterval: This option specifies the page count interval.
  5. DOSSiteInterval: This option specifies the site count interval.
  6. DOSBlockingPeriod: This option defines the amount of time in seconds that a client will be blocked.
  7. DOSEmailNotify: This option sends an email to the specified address when an IP address has been blacklisted.
  8. DOSSystemCommand: Whenever an IP address has been blacklisted, the specified system command will be executed.
  9. DOSLogDir: This option defines the mod_evasive log directory.

A script to test mod_evasive can be invoked by the command:

perl /usr/share/doc/libapache2-mod-evasive/examples/test.pl

As we have specified a log file and directory for mod_evasive if would be a good idea to actually provide that space.

sudo mkdir /var/log/apache2/mod_evasive
sudo chown –R www-data:www-data /var/log/apache2/mod_evasive

More information on mod_evasive can be found on the authors github page.

One can disable mod_evasive by editing the /etc/apache2/apache2.conf file and disabling the line “#AddModule mod_evasive.c” and restarting the apache service.

Published by crawls