Best practices

When securing web applications we recommend a bottom-up approach, see level 1 (lowest) to level 4 features within the chapter Security feature checklist. This approach has the benefit of quickly achieving a high level of security while requiring low effort.
Begin by configuring HTTP protocol validation and limitation. This is generally not application-specific and requires little effort and testing.
Proceed by activating URL-signing and form-signing. This step also requires only little configuration effort, and should work fine as long as the HTML generated by your web application is not severely broken. Initially, it may be useful not to block unsigned URL and form requests (instead, an error is traced to the log) to identify places where the signing has failed (i.e. JavaScript, or broken HTML).
As a final measure, apply input validation on HTML forms by specifying the types and ranges (through regular expressions) of the form fields. This step requires considerable configuration and testing effort, and is application-specific, but very effective against injection attacks.

Content filtering should always be combined with an ErrorFilter (see the chapter ErrorFilter) showing an appropriate error message when denying a request as well as with an HTTP request validation filter, e.g., by using the HTTP-Validation-Length profile, to limit the HTTP request size processed by nevisProxy.

A sample web application

For the following sections, assume a sample web application with the following URI namespace:

/
/js/*                    JavaScript
/css/*                   Stylesheets
/img/*                   Images
/appl/static/*           Static HTML pages
/appl/servlet/*          Dynamically generated content

Configuring HTTP protocol validation and limitation

HTTP protocol validation and limitation is not application-specific and generally does not lead to integration problems. You configure HTTP protocol validation and limitation by adding a ModsecurityFilter with the OWASP Core Rule Set, which contains rules for protocol enforcement as well as rules against protocol attacks. These rules will block requests that do not meet the HTTP RFC requirements, or that attack the HTTP protocol, such as HTTP Request Smuggling and Response Splitting. Map the filter close to the root of the application, so that invalid requests will be blocked before they reach sensitive parts. See the chapter ModSecurity Configuration Guide on how to setup the ModSecurityFilter.

Configuring URL signing and form signing

To protect against cross-site request forgery (CSRF) and forceful browsing attacks, you can enable URL- and Form-Signing by configuring an EncryptionFilter.

<filter>
 <filter-name>SigningFilter</filter-name>
 <filter-class>ch::nevis::isiweb4::filter::validation::EncryptionFilter</filter-class>
 <init-param>
          <param-name>ErrorPolicy</param-name>
          <param-value>
                   PLAIN:block:403
                   UNKNOWN_KEY:block:403
                   FAILED:block:403
          </param-value>
 </init-param>
 <init-param>
          <param-name>URLMode</param-name>
          <param-value>hmacsha1</param-value>
 </init-param>
 <init-param>
          <param-name>FormMode</param-name>
          <param-value>formsig</param-value>
 </init-param>
 <init-param>
          <param-name>EntryURL</param-name>
          <param-value>
                   /appl/
                   /appl/bookmarkable-url/
          </param-value>
 </init-param>
 <init-param>
          <param-name>GlobalRandom</param-name>
          <param-value>blwgfcu3owghc9743pyrdfhx43uewhgc963w</param-value>
          <description>Defines a random seed used for HMAC tagging any signed or encrypted URL.</description>
 </init-param>
</filter>

Here we have configured an EncryptionFilter that will sign links and forms in the HTML content. We have configured two entry URLs, these can be accessed unsigned (entry page or bookmarkable page). Otherwise, unsigned requests are blocked. See the chapter EncryptionFilter for a detailed list of all the options.

It can be useful, especially in the beginning, to enable URL signing but only trace an error to the log if an unsigned request arrives. This can be configured using an ErrorPolicy of TRACE. The logs can then provide hints as to which URLs are being accessed without signature (e.g. bookmarks). Also it might reveal that the offending request originated from a JavaScript action or broken HTML, which could not be successfully parsed and rewritten.

Configuring input validation

For fine-grained input validation you can configure a HeaderValidationFilter for request and response headers, or a ParameterValidationFilter that restricts the set of values your web application can receive (for example, from form submissions).

An example web application may contain a form where the user can enter an account number of the form "aa-nnn", and a numerical amount to transfer. Any other input is disallowed. You can implement this use case with the ParameterValidationFilter:

<filter>
    <filter-name>ParameterValidationFilter</filter-name>
    <filter-class>ch::nevis::nevisproxy::filter::validation::ParameterValidationFilter</filter-class>
    <init-param>
        <param-name>ParameterRules</param-name>
        <param-value>
               ^accountno$:^[a-zA-Z][a-zA-Z]-[0-9][0-9][0-9]$:allow
               ^accountno$:.*:deny
               ^amount$:^[0-9]*$:allow
               ^amount$:.*:deny
        </param-value>
    </init-param>
</filter>
       
<filter-mapping>
      <filter-name>ParameterValidationFilter</filter-name>
      <url-pattern>/*</url-pattern>
</filter-mapping>

This type of fine-grained input validation is very powerful, but requires you to have detailed knowledge of your web application. Also the configuration has to be kept in sync with any updates in the application, see the chapter Self-learning of input validation below.

Self-learning of input validation

The InputValidationFilter provides a learning mechanism to generate the ParameterRules patterns, which are used to protect an application.

This learning mechanism is based on two components:

• The first component is an audit log that contains the data transferred from clients to the server. This audit log is generated by the InputValidationFilter that has been mapped to the application's URL.
• The second component is a rule generator that processes the audit log data and creates ParameterRules rules matching the captured data.

The following steps are necessary to learn a new rule set:

1. Add an InputValidationFilter and map it to the URL of the application. Configure the audit.log facility of the filter. You also need to specify the decoding you want to apply to the incoming data. It is important to use exactly the same decoding during the learning phase that you are going to use later when the rules are enforced.

Example filter configuration:

<filter>
  <filter-name>SelfLearningValidationFilter</filter-name>
  <filter-class>ch::nevis::isiweb4::filter::validation::InputValidationFilter</filter-class>
  <init-param>
    <param-name>AuditLog.FileName</param-name>
    <param-value>/var/opt/nevisproxy/<instance_name>/logs/test_input_audit.log</param-value>
  </init-param>
  <init-param>
    <param-name>AuditLog.PassPhrase</param-name>
    <param-value>${exec: cat /var/opt/neviskeybox/default/public/node_keypass}</param-value>
  </init-param>
  <init-param>
    <param-name>DecodingRules</param-name>
    <param-value>URL_decode_unicode</param-value>
  </init-param>
</filter>
 
<filter-mapping>
  <filter-name>SelfLearningValidationFilter</filter-name>
  <url-pattern>/appl/servlet/</url-pattern>
</filter-mapping>

2. Run the rule generator multiple times, for example every hour. The rule generator processes the audit log as well as the existing rules and updates them. Repeat this process multiple times (usually for several days) until the rules do not change anymore.

You can invoke the rule generator with the nevisproxy whitelist command, which has the following syntax:

Usage:
  nevisproxy <instance> whitelist <application> <passphrase> <audit log> <rule file> ['utf8'] ['varnames']
 
Options:
 application: name of the rule set to generate
 passphrase:  the passphrase to decrypt the data (or a program writing the passphrase to stdout)
 audit log:   audit log file to read the collected data from
 rule file:   rule file to read/write the generated rules
 utf8:        enables utf8 multibyte character support
 varnames:    generates ParameterRules rules with variable parameter names

The next code block gives an example of the nevisproxy whitelist command:

nevisproxy siven whitelist test-profile some_pwd /var/opt/nevisproxy/siven/logs/test_input_audit.log /var/opt/nevisproxy/siven/conf/profile.properties varnames

The command generates a file called profile.properties. This file contains the profile "test-profile" with parameter rules based on the content of the test_input_audit.log file:

# profile 'test-profile' implementing input validation
# statistics 'rule.'<id>'='<name>':'<total updates>':'<total hits>' '<updates>':'<hits>
# rule.0=test_param_1:2:18 0:6
# rule.1=test_param_2:2:18 0:6
# rule.2=test_param_3:1:4 0:2
ch.nevis.isiweb4.filter.validation.test-profile.Description=input validation, generated Mon Apr 20 13:32:08 2020, 0 updates
ch.nevis.isiweb4.filter.validation.test-profile.BlockOnError=true
ch.nevis.isiweb4.filter.validation.test-profile.RegexpType=PCRE
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.0=test_param_1:^[a-zA-Z0-9]{6,13}$:allow:silent
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.1=test_param_1:.*:deny
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.2=test_param_2:^[a-zA-Z0-9]{8,31}$:allow:silent
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.3=test_param_2:.*:deny
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.4=test_param_3:^[\?]{0,1}$:allow:silent
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.5=test_param_3:.*:deny
ch.nevis.isiweb4.filter.validation.test-profile.ParameterRules.6=.*:.*:allow:log

3. Once the rules no longer change after you invoked the rule generator, you can upload the profile into the InputValidationFilter (the audit log configuration is no longer necessary):

<filter>
  <filter-name>SelfLearningValidationFilter</filter-name>
  <filter-class>ch::nevis::isiweb4::filter::validation::InputValidationFilter</filter-class>
  <init-param>
    <param-name>Profile</param-name>
    <param-value>test-profile</param-value>
  </init-param>
</filter>

4. Restart the proxy instance and monitor the navajo.log file. Update the rule set if there are any false rule violations (false positives) within the navajo.log file.

Where does the InputValidationFilter learn?

The InputValidationFilter learns rules by activating the audit log either within a production or test environment:

Environment	Advantages	Disadvantages
Production	- You can invoke the rule generator until there are no further updates to the rules. - You can directly activate the new profile.- The generator can learn from real data that provides an accurate rule set.	- There is the risk of learning rules from invalid user input or even from an attempted attack.
Test	- There is no risk of learning rules from an invalid pattern or potential attack.	- You need to define extensive test patterns to learn from. - You need to test the generated profile in the production environment in permissive mode (BlockOnError=false).

URI whitelisting

A whitelist of all allowed URIs (path portion) may be generated based on existing access.log files. These rules are used to restrict access to known handlers of the application only, and may be used in addition to the parameter whitelisting described in the previous section, Self-learning of input validation. The following example demonstrates a shell script which generates/updates a request path whitelisting.

#!/bin/sh
PATH=/usr/bin:/bin:/opt/nevisproxy/bin
 
# the OPTIONS variable controls the rule generation, enter "nevisproxy default genrules"
 
# for a list of available options
OPTIONS="-h"
 
# PFX is the name (path portion) of your application
PFX="appl"
 
 
# this sample script uses the "default" proxy instance
 
# the log file contains all relevant URIs and is used every time you update the rules
LOG=/var/opt/nevisproxy/default/logs/$PFX.log
 
# the rule db file contains the URI pattern you may edit
DB=/var/opt/nevisproxy/default/logs/$PFX.db
TMP=$DB.tmp
 
# filters the access.log file by the defined application path /$PFX cat /var/opt/nevisproxy/default/logs/access.log | \
cut -d ' ' -f 7 | cut -d '?' -f 1 | grep "^/$PFX/" > $LOG.tmp
touch $DB
touch $LOG
cat $LOG >> $LOG.tmp
nevisproxy default genrules -i $LOG.tmp -c $DB $OPTIONS > $TMP
rm -f $LOG.tmp
grep "# ADD line " $TMP | cut d ' ' -f 5 > $LOG
grep "^QS_PermitUri +QSF" $TMP > $DB
rm -f $TMP
 
echo "ch.nevis.isiweb4.filter.validation.$PFX.Description=generated for /$PFX"
echo "ch.nevis.isiweb4.filter.validation.$PFX.BlockOnError=true"
echo "ch.nevis.isiweb4.filter.validation.$PFX.DecodingRules=URL_decode_unicode"
echo "ch.nevis.isiweb4.filter.validation.$PFX.RegexpType=PCRE"
cat $DB | awk '{print substr($0, 27, length($0)-27)}' | \
    awk -F'"' '{ print "ch.nevis.isiweb4.filter.validation..URIWhiteList." NR-1 "=" $2}'| \
    sed -e "s::$PFX:g"

Such a script may be executed multiple times until it has learned all possible URIs and the output (stdout) can be added to the customer profiles to be mapped to the application which has been defined by the PFX variable within the script profile, see the chapter "Managing custom profiles.

HTTP security headers

Browser developers have integrated features to counter different types of attacks. As attacks often exploit the fact that clients trust the web server and the content received from it, optional HTTP protocol headers were introduced. These headers allow the application to propagate its intentions to the browser. This information enables the browser to detect discrepancies between the policies (what the application had been intending to send) and the content of the HTTP response body (what the browser actually receives), such as JavaScript code or references to other websites.

List of optional HTTP headers and their advantages:

• X-Frame-Options
  The X-Frame-Options HTTP response header specifies whether to allow a browser to render a page in a frame (<frame>), inline frame (<iframe>) or object (<object>). Sites can use the header to avoid clickjacking attacks, by ensuring that the site content is not embedded into other sites. It also helps against various other attacks, such as drag and drop cross-site scripting (XSS), copy and paste XSS, invisible sitewide XSS, docmode downgrades, frame busting as well as side channels and cross-origin leaks. Example: X-Frame-Options: SAMEORIGIN
• Strict-Transport-Security
  The server can advise the browser to interact exclusively via HTTPS by specifying a period of time during which the browser shall access the server in a secure fashion, that is, via HTTPS only. This prevents a fallback to an insecure HTTP connection in the case where an HTML page contains non-HTTPS references. It also helps to detect man-in-the-middle attacks where a proxy between the client and server converts an HTTPS to an HTTP connection. It is supported by all browsers (except for Opera Mini). Example: Strict-Transport-Security: max-age=63072000;
• X-Content-Type-Options
  Setting X-Content-Type-Options to "nosniff" prevents Internet Explorer from MIME-sniffing a response away from the declared content type. This helps prevent media types from being interpreted as HTML or JavaScript code, e.g., a JavaScript XSS vector that has been uploaded as an image to a Wiki server. Example:X-Content-Type-Options: nosniff*
• Content-Security-Policy
  This header enables the specification of complex policies that describe allowed actions in detail. See the "getting started" at the Mozilla Developer Network for further information resp. the CSP specification at w3.org. Example: Content-Security-Policy: default-src 'self'

The following example illustrates how you can add these security headers to HTTP responses using a DelegationFilter within the nevisProxy configuration.

<filter>
<filter-name>SecurityHeaders</filter-name>
<filter-class>ch::nevis::isiweb4::filter::delegation::HeaderDelegationFilter</filter-class>
<init-param>
<param-name>DelegateToFrontend</param-name>
<param-value>
Strict-Transport-Security:CONST:max-age=63072000
X-Frame-Options:CONST:SAMEORIGIN
X-Content-Type-Options:CONST:nosniff
Content-Security-Policy:CONST:default-src 'self'
</param-value>
</init-param>
</filter>