 > ModSecurity Handbook: Getting Started: Chapter 1. Introduction |  |
> ModSecurity Handbook: Getting Started: Chapter 1. Introduction | |
ModSecurity is a tool that will help you secure your web applications—no, scratch that: ModSecurity is a tool that will help you sleep better at night; in this book, we’ll explain how. We usually call ModSecurity a web application firewall (WAF), the generally accepted term to refer to the class of products designed specifically to secure web applications. Other times, we call it an HTTP intrusion detection tool, because we think that name better describes what ModSecurity does. Neither name is entirely adequate, but we don’t have a better one. However, it doesn’t really matter what we call it. The point is that web applications—yours, ours, everyone’s—are terribly insecure on average. We all struggle to keep ahead of security issues and need any help we can get to handle them.
Ivan thought to create ModSecurity while he was responsible for the security of several web-based products. He could see how insecure most web applications were, slapped together with little time spent on design and even less time spent on understanding security issues. Not only were web applications insecure, but people generally had little awareness of whether they were being attacked or exploited. Most web servers kept only standard access and error logs, and they didn’t say much.
ModSecurity will help you sleep better at night because, above all, it solves the visibility problem: it lets you see your web traffic. That visibility is key to security; once you can see HTTP traffic, you can analyze it in real time, record it as necessary, and react to the events. The best part of this concept is that you get to do all of that without actually touching web applications. Even better, the concept can be applied to any application—even if you can’t access its source code.
Like many other open source projects, ModSecurity started out as a hobby. Back in 2002, producing secure web applications was virtually impossible. (It’s the same these days, sadly.) However, that realization led to the idea of a tool that would sit in front of web applications and control the flow of data to and from the system. The first version was released in November 2002, but a few more months were needed before the tool became useful. Other people started to learn about ModSecurity, and its popularity started to rise.
Initially, most development effort for the tool went into wrestling with Apache to make request body inspection possible. Apache 1.3.x didn’t include any interception or filtering APIs, but it was still possible to trick it into submission. Apache 2.x improved the situation by providing APIs that allowed content interception, but no documentation was available.
By 2004, Ivan converted from obsessing about software development to obsessing about web application security. He quit his job and started treating ModSecurity as a business. In the summer of 2006, ModSecurity went head-to-head with other web application firewalls in an evaluation conducted by Forrester Research, and it achieved great results. Later that year, ModSecurity was acquired by Breach Security. A team of one eventually became a team of many: Brian Rectanus came to work on ModSecurity, Ofer Shezaf embarked on the rules, and Ryan C. Barnett handled community management and education. ModSecurity 2.0, a complete rewrite, was released in late 2006. Breach Security also released ModSecurity Community Console, which combined the functionality of a remote logging sensor and a monitoring and reporting GUI.
In 2009, Ivan left Breach Security. He stayed involved with ModSecurity for a while, but mostly worked on the first edition of this book. In his own words, he couldn’t leave the project if it wasn’t properly documented. Brian Rectanus took the lead. In the meantime, Ryan C. Barnett took charge of the ModSecurity rules and produced significant improvements with Core Rule Set v2. In 2010, Trustwave acquired Breach Security and promised to revitalize ModSecurity. The project was then handed to Ryan C. Barnett and Breno Silva.
Something remarkable happened in March 2011: Trustwave announced that it would change the license of ModSecurity from GPLv2 to Apache Software License (ASLv2). This was a great step toward a wider use of ModSecurity because ASL falls into the category of permissive licenses. Later, the same change was announced for the Core Rule Set project, which is hosted with the Open Web Application Security Project (OWASP). Subsequently, commercial WAF offerings started to incorporate the ModSecurity engine and added the OWASP ModSecurity Core Rules as a default ruleset. With version 2.7.0, ModSecurity was ported to work with Nginx and IIS web servers, but these ports never achieved the stability of the original version. This eventually led to a major rewrite that would be able to support multiple platforms equally well. That will become ModSecurity 3.0, currently in the making.
In 2013, Felipe Costa took over the lead developer position from Breno, and when Ryan left Trustwave in 2015 he handed over the rules to Chaim Sanders, who joined Trustwave in 2014 to support the project with coding and community management.
ModSecurity is a toolkit for real-time web application monitoring, logging, debugging, and access control. I like to think of it as an enabler. There are no hard rules telling you what to do; instead, it’s up to you to choose your own path through the available features. That’s why the title of this section asks what ModSecurity can do, not what it does.
The freedom to choose what to do is an essential part of ModSecurity’s identity and goes well with its open source nature. With full access to the source code, your freedom to choose extends to the ability to customize and extend the tool itself to make it fit your needs. This is a matter not of ideology, but of practicality. I simply don’t want my tools to restrict what I can do.
The following is a list of the most important usage scenarios for ModSecurity:
At its core, ModSecurity gives you access to the HTTP traffic stream in real time, along with the ability to inspect it. This is enough for real-time security monitoring. There’s an added dimension of what’s possible through ModSecurity’s persistent storage mechanism, which enables you to track system elements over time and perform event correlation. You can block reliably, if you so wish, because ModSecurity uses full request and response buffering.
Virtual patching is a concept that addresses vulnerability mitigation in a separate layer, in which you get to fix problems in applications without having to touch the applications themselves. Virtual patching is applicable to applications that use any communication protocol, but it’s particularly useful with HTTP, because traffic generally can be well understood by an intermediary device. ModSecurity excels at virtual patching because of its reliable blocking capabilities and the flexible rule language that can be adapted to any need. Virtual patching is, by far, the activity ModSecurity offers that requires the least investment, is the easiest to perform, and that most organizations can benefit from straight away.
Web servers traditionally do very little when it comes to logging for security purposes. They log very little by default, and even with a lot of tweaking you can’t get all the data that you need. I have yet to encounter a web server that is able to log full transaction data—but ModSecurity gives you the ability to log everything, including raw transaction data, which is essential for forensics. In addition, you get to choose which transactions are logged, which parts of a transaction are logged, and which parts are sanitized. As a bonus, this type of detailed logging is also helpful for application troubleshooting—not just security.
Security assessment is seen largely as an active scheduled event, in which an independent team is sourced to try to perform a simulated attack. Continuous passive security assessment is a variation of real-time monitoring in which instead of focusing on the behavior of the external parties, you focus on the behavior of the system itself. It’s an early warning system of sorts that can detect traces of many abnormalities and security weaknesses before they are exploited.
One of my favorite uses for ModSecurity is attack surface reduction, in which you selectively narrow down the HTTP features you’re willing to accept (e.g., request methods, request headers, content types, etc.). ModSecurity can assist you in enforcing many similar restrictions, either directly or through collaboration with other Apache modules. For example, it’s possible to fix many session management issues, as well as cross-site request forgery vulnerabilities.
Real life often makes unusual demands of us, and when handling such demands, the flexibility of ModSecurity comes in handy when you need it the most. You may have to address a security need, or maybe you have a completely different issue; for example, some people use ModSecurity as an XML web service router, combining its ability to parse XML and apply XPath expressions with its ability to proxy requests. Who knew?
I’m often asked if ModSecurity can be used to protect Apache itself. The answer is that it can, in some limited circumstances, but that it isn’t what it’s designed for. You may sometimes be able to catch an attack with ModSecurity before it hits a vulnerable spot in Apache or in a third-party module, but there’s a large quantity of code that runs before ModSecurity. If there’s a vulnerability in that area, ModSecurity won’t be able to do anything about it.
There are three guiding principles on which ModSecurity is based:
ModSecurity was designed and built with a particular user in mind: a security expert who needs to be able to intercept, analyze, and store HTTP traffic. I didn’t see much value in hard-coded functionality, because real life is so complex that everyone needs to do things just slightly differently. ModSecurity achieves flexibility by providing a powerful rule language, which allows you to do exactly what you need to, in combination with the ability to apply rules only where you need to: granular control down to the individual byte.
Another key design decision was to make ModSecurity as passive as possible; it will thus never make changes to transaction data unless instructed to do so. The key reason for this was to give users confidence to deploy ModSecurity with entirely passive rulesets that allow them to just observe, safe in knowing that their applications will not be affected. That’s why ModSecurity will give you plenty of information, but ultimately leave the decisions to you.
There’s no such thing as a perfect tool, but a predictable one is the next best thing. Armed with all the facts, you can understand ModSecurity’s weak points and work around them.
There are elements in ModSecurity that fall outside the scope of these principles. For example, ModSecurity can change the way Apache identifies itself to the outside world, confine the Apache process within a jail, and even inject security tokens into the traffic. Although these functions are useful, I think that they detract from the main purpose of ModSecurity, which is to be a reliable and predictable tool that enables HTTP traffic inspection.
ModSecurity supports two deployment options: embedded and reverse proxy deployment. There is no one correct way to use them; choose an option based on what best suits your circumstances. There are advantages and disadvantages of both options:
Because ModSecurity is an Apache module, you can add it to any compatible version of Apache. At the moment, that means a reasonably recent Apache version, ideally from the 2.4.x branch. That said, a version from the 2.2.x branch will also work. ModSecurity has been ported to Nginx and to IIS, which introduces wider platform options. The embedded option is a great choice for those who already have their architecture laid out and don’t want to change it. Embedded deployment is also the preferred option if you need to protect hundreds of web servers. In such situations, it is impractical to build a separate proxy-based security layer. Embedded ModSecurity not only does not introduce new points of failure, but it scales seamlessly as the underlying web infrastructure scales. The main challenge of embedded deployment is that server resources are shared between the web server and ModSecurity.
Reverse proxies are effectively HTTP routers, designed to stand between web servers and their clients. When you install a dedicated Apache reverse proxy and add ModSecurity to it, you get a “proper” network web application firewall, which you can use to protect any number of web servers on the same network. Many security practitioners prefer having a separate security layer, with which you get complete isolation from the systems you are protecting. On the performance front, a standalone ModSecurity installation will have resources dedicated to it, which means that you will be able to do more (i.e., have more complex rules). The main disadvantage of this approach is the new point of failure, which will need to be addressed with a high-availability setup of two or more reverse proxies.
In this first practical section of the book, I will give you a whirlwind tour of ModSecurity’s internals to help you get started.
ModSecurity is a hybrid WAF engine that relies on the host web server for some of its work. ModSecurity was originally written for the Apache web server but has since been ported to Nginx and to IIS. Although both ports are actively maintained, they suffer from ModSecurity’s heritage and tight integration with the Apache source code. The next major version of ModSecurity is being reimplemented to separate it from Apache, allowing it to support all web servers equally well. Until that happens, the best web server to run ModSecurity is Apache 2.x.
Apache does for ModSecurity what it does for all other modules—it handles the following infrastructure tasks:
There are a few additional tasks Apache performs in a reverse proxy scenario:
The advantage of a hybrid implementation is that it’s efficient; the duplication of work is minimal when it comes to HTTP parsing. A couple of disadvantages of this approach are that you don’t always get access to the raw data stream and that web servers sometimes don’t process data in the way a security-conscious tool would. In the case of Apache, the hybrid approach works reasonably well, with a few minor issues:
This normally isn’t a problem, because what Apache doesn’t see can’t harm any module or application. In some rare cases, however, the purpose of NUL-byte evasion is to hide something, and this Apache behavior only helps with the hiding.
Apache will canonicalize request headers, combining multiple headers that use the same name and collapsing those that span two or more lines. The transformation may make it difficult to detect subtle signs of evasion, but in practice this hasn’t been a problem yet.
Apache will handle some requests quickly, leaving ModSecurity unable to do anything but notice them in the logging phase. Invalid HTTP requests, in particular, will be rejected by Apache without ModSecurity having a say.
Because of the way Apache works, the Server and
Date response headers are invisible to ModSecurity in embedded
mode; they can’t be inspected or logged.
The functionality offered by ModSecurity falls roughly into four areas:
ModSecurity tries to make sense of as much data as available. The supported data formats are backed by security-conscious parsers that extract bits of data and store them for use in the rules.
In a typical installation, both request and response bodies will be buffered. This means ModSecurity usually sees complete requests before they’re passed to the application for processing, and complete responses before they’re sent to clients. Buffering is an important feature, because it’s the only way to provide reliable blocking. The downside of buffering is that it requires additional RAM to store the request and response body data.
Full transaction logging (also referred to as audit logging) is a big part of what ModSecurity does. This feature allows you to record complete HTTP traffic instead of just rudimentary access log information. Request headers, request body, response header, response body—all those bits will be available to you. It is only with the ability to see what’s happening that you will be able to stay in control.
The rule engine builds on the work performed by all other components. By the time the rule engine starts operating, the various bits and pieces of data it requires will all be prepared and ready for inspection. At that point, the rules will take over to assess the transaction and take actions as necessary.
There’s one thing ModSecurity purposefully avoids doing: as a matter of design, ModSecurity does not support data sanitization. I don’t believe in sanitization, purely because I believe that it is too difficult to get right. If you know for sure that you’re being attacked (as you have to before you can decide to sanitize), then you should refuse to process the offending requests altogether. Attempting to sanitize merely opens a new battlefield in which your attackers don’t have anything to lose but have everything to win. You, on the other hand, don’t have anything to win but everything to lose.
Every part of ModSecurity revolves around two things: configuration and rules. The configuration tells ModSecurity how to process the data it sees; the rules decide what to do with the processed data. Although it’s too early to go into how the rules work, I’ll include a quick example here just to give you an idea of what they look like. For example:
SecRule ARGS "@rx <script>" \
"id:2000,log,deny,status:404"Even without further assistance, you can probably recognize the part in
the rule that specifies what we want to look for in input data
(<script>). Similarly, you’ll easily figure out what will happen
if we do find the desired pattern (log,deny,status:404). Things will
become more clear when you look at the general rule syntax, as follows:
SecRule VARIABLES OPERATOR ACTIONS
The three parts have the following meanings:
The VARIABLES part tells ModSecurity where to
look. The ARGS variable, used in the example, indicates all request
parameters.
The OPERATOR part tells ModSecurity how to look.
In the example, we have a regular expression pattern, which will be matched against
ARGS.
The ACTIONS part is used to add metadata to the
rules and to specify what ModSecurity should do when a match occurs. The rule from the
previous example assigns ID 2000 to uniquely identify the rule and specifies the
following actions on a match: log problem, stop transaction processing, and return HTTP
response code 404.
I hope you aren’t disappointed with the simplicity of this first rule. I promise you that by combining the various facilities offered by ModSecurity, you will be able to write useful rules that implement complex logic where necessary.
In ModSecurity, every transaction goes through five steps, or phases. In each of the phases, ModSecurity will perform some work at the beginning (e.g., parse data that has become available), invoke the rules specified to work in that phase, and perhaps perform a task or two after the phase rules have finished. At first glance, it may seem that five phases are too many, but there’s a reason that each phase exists. There is always one task, sometimes several, that can only be performed at a particular moment in the transaction lifecycle.
The request headers phase is the first entry point for ModSecurity. The principal purpose of this phase is to allow rule writers to assess a request before the costly request body processing is undertaken. Similarly, there is often a need to influence how ModSecurity will process a request body, and in this phase is the time to do it. For example, ModSecurity will not parse an XML or JSON request body by default, but you can instruct it do so by placing the appropriate rules into phase 1.
The request body phase is the main request analysis phase and takes place immediately after a complete request body has been received and processed. The rules in this phase have all the available request data at their disposal. Afterward, the web server will either generate the response itself (in embedded mode) or forward the transaction to a backend web server (in reverse proxy mode).
The response headers phase takes place after response headers become available but before a response body is read. The rules that need to decide whether to inspect a response body should run in this phase.
The response body phase is the main response analysis phase. By the time this phase begins, the response body will have been read and all its data made available for the rules to make their decisions.
The logging phase is special. It’s the only phase from which you cannot block. By the time this phase runs, the transaction will have finished, so there’s little you can do but record the fact that it happened. Rules in this phase are run to control how logging is performed or to save information in persistent storage.
To give you a better idea of what happens in every transaction, we’ll examine a detailed debug log of one POST transaction. The debug log is an additional logging facility provided by ModSecurity that allows you to observe the execution steps of the module in great detail. I’ve deliberately chosen a transaction type that uses the request body as its principal method to transmit data, because such a transaction will exercise most parts of ModSecurity. To keep things relatively simple, I used a configuration without any rules, removed some of the debug log lines for clarity, and removed the timestamps and some additional metadata from each log line.
Please don’t try to understand everything about the logs at this point. The idea is to get a general feel for how ModSecurity works and an introduction to debug logs. Soon after you start to use ModSecurity, you’ll discover that debug logs are an indispensable rule-writing and troubleshooting tool.
The transaction I’m using as an example in this section is very straightforward. I made a point of placing request data in two different places—parameter a in the query string and parameter b in the request body—but there’s little else of interest in the request:
POST /?a=test HTTP/1.0 Content-Type: application/x-www-form-urlencoded Content-Length: 6 b=test
The response is entirely unremarkable:
HTTP/1.1 200 OK Date: Fri, 22 Jul 2016 04:59:13 GMT Server: Apache Content-Length: 12 Connection: close Content-Type: text/html Hello World!
ModSecurity is first invoked by Apache after request headers become
available but before a request body (if any) is read. First comes the initialization
message, which contains the unique transaction ID generated by
mod_unique_id. With this, you should be able to pair the information
in the debug log with the information in your access and audit logs. At this point,
ModSecurity will parse the information in the request line and in the request headers. In
this example, the query string part contains a single parameter (a), so
you’ll see a message documenting its discovery. ModSecurity will then create a transaction
context and invoke the REQUEST_HEADERS phase:
[4] Initialising transaction (txid V5LjWH8AAQEAAFPTr64AAAAA). [5] Adding request argument (QUERY_STRING): name "a", value "test" [4] Transaction context created (dcfg 1154668). [4] Starting phase REQUEST_HEADERS.
Assuming that a rule didn’t block the transaction, ModSecurity will now return control to Apache, allowing other modules to process the request before control is given back to it.
In the second phase, ModSecurity will first read and process the
request body, if it’s present. In the following example, you can see three messages from
the input filter, which tell you what was read. The fourth message tells you that one
parameter was extracted from the request body. The content type used in this request
(application/x-www-form-urlencoded) is one of the types ModSecurity
recognizes and parses automatically. Once the request body is processed, the
REQUEST_BODY rules are processed.
[4] Second phase starting (dcfg 1154668). [4] Input filter: Reading request body. [9] Input filter: Bucket type HEAP contains 6 bytes. [9] Input filter: Bucket type EOS contains 0 bytes. [5] Adding request argument (BODY): name "b", value "test" [4] Input filter: Completed receiving request body (length 6). [4] Starting phase REQUEST_BODY.
The filters mentioned in the logs are parts of ModSecurity that handle request and response bodies:
[4] Hook insert_filter: Adding input forwarding filter (r 7f5fc8002970). [4] Hook insert_filter: Adding output filter (r 7f5fc8002970).
There will be a message in the debug log every time ModSecurity sends a chunk of data to the request handler, and one final message to say that there isn’t any more data in the buffers:
[4] Input filter: Forwarding input: mode=0, block=0, nbytes=8192 (f 7f5fc800ae90, r 7f5fc8002970). [4] Input filter: Forwarded 6 bytes. [4] Input filter: Sent EOS. [4] Input filter: Input forwarding complete.
The request is now in the hands of Apache’s request handler. If the web server is running in embedded mode, the request handler will generate the response itself. If it’s running in reverse proxy mode, the server will forward the transaction to a backend server.
Shortly thereafter, the output filter will start receiving data, at
which point the RESPONSE_HEADERS rules will be invoked:
[9] Output filter: Receiving output (f 7f5fc800aeb8, r 7f5fc8002970). [4] Starting phase RESPONSE_HEADERS.
Once all the rules have run, ModSecurity will continue to store the
response body in its buffers, after which it will run the RESPONSE_BODY
rules:
[9] Content Injection: Not enabled. [9] Output filter: Bucket type MMAP contains 13 bytes. [9] Output filter: Bucket type EOS contains 0 bytes. [4] Output filter: Completed receiving response body (buffered full - 12 bytes). [4] Starting phase RESPONSE_BODY.
Again, assuming that none of the rules blocked, the accumulated response body will be forwarded to the client:
[4] Output filter: Output forwarding complete.
Finally, the logging phase will commence. The
LOGGING rules will be run first to allow them to influence logging,
after which the audit logging subsystem will be invoked to log the transaction if
necessary. A message from the audit logging subsystem will be the last transaction message
in the logs. In this example, ModSecurity tells us that it didn’t find anything of
interest in the transaction and that it sees no reason to log it:
[4] Initialising logging. [4] Starting phase LOGGING. [4] Recording persistent data took 0 microseconds. [4] Audit log: Ignoring a non-relevant request.
Requests that contain files are processed slightly differently. The changes can be best understood by again following the activity in the debug log:
[4] Input filter: Reading request body. [9] Multipart: Boundary: ------------------------ce3de83f6cf79943 [9] Input filter: Bucket type HEAP contains 140 bytes. [9] Multipart: Added part header "Content-Disposition" "form-data; name=\"f\"; filename=\"eicar.com.txt\"" [9] Multipart: Added part header "Content-Type" "text/plain" [9] Multipart: Content-Disposition name: f [9] Multipart: Content-Disposition filename: eicar.com.txt [9] Input filter: Bucket type HEAP contains 116 bytes. [4] Multipart: Created temporary file 1 (mode 0600): /usr/local/modsecurity/var/tmp/20160723-054018-V5LnIn8AAQEAAFurQfQAAAAA-file-F7zAlU [9] Multipart: Added file part 7f67b400fd50 to the list: name "f" file name "eicar.com.txt" (offset 140, length 68) [9] Input filter: Bucket type EOS contains 0 bytes. [4] Request body no files length: 96 [4] Input filter: Completed receiving request body (length 256).
In addition to seeing the multipart parser in action, you’ll see ModSecurity creating a temporary file (into which it will extract the upload) and adjusting its privileges to match the desired configuration.
Then, at the end of the transaction, you’ll see the cleanup and the temporary file deleted:
[4] Multipart: Cleanup started (remove files 1). [4] Multipart: Deleted file (part) "/usr/local/modsecurity/var/tmp/20160723-054427-V5LoG38AAQEAAF4SFAoAAAAT-file-bwi2wv"
The temporary file won’t be deleted if ModSecurity decides to keep an uploaded file. Instead, it will be moved to the storage area:
[4] Multipart: Cleanup started (remove files 0). [4] Input filter: Moved file from "/usr/local/modsecurity/var/tmp/20160723-054018-V5LnIn8AAQEAAFurQfQAAAAA-file-F7zAlU" to "/usr/local/modsecurity/var/upload/20160723-054018-V5LnIn8AAQEAAFurQfQAAAAA-file-F7zAlU".
In the example traces, you’ve observed an upload of a small file that was stored in RAM. When large uploads take place, ModSecurity will attempt to use RAM at first, switching to on-disk storage once it becomes obvious that the file is larger:
[9] Input filter: Bucket type HEAP contains 6080 bytes. [9] Input filter: Bucket type HEAP contains 2112 bytes. [9] Input filter: Bucket type HEAP contains 5888 bytes. [9] Input filter: Bucket type HEAP contains 2304 bytes. [9] Input filter: Bucket type HEAP contains 5696 bytes. [9] Input filter: Bucket type HEAP contains 2496 bytes. [9] Input filter: Bucket type HEAP contains 5504 bytes. [9] Input filter: Bucket type HEAP contains 2688 bytes. [9] Input filter: Bucket type HEAP contains 5312 bytes. [9] Input filter: Bucket type HEAP contains 2880 bytes. [9] Input filter: Bucket type HEAP contains 5120 bytes. [9] Input filter: Bucket type HEAP contains 3072 bytes. [4] Input filter: Request too large to store in memory, switching to disk.
A new file will be created to store the entire raw request body:
[4] Input filter: Created temporary file to store request body: /usr/local/modsecurity/var/tmp/20160723-054813-V5Lo-X8AAQEAAF4SFAsAAAAQ-request_body-pJs7UV [4] Input filter: Wrote 128146 bytes from memory to disk.
This file is always deleted in the cleanup phase:
[4] Multipart: Deleted file (part) "/usr/local/modsecurity/var/tmp/20160723-054813-V5Lo-X8AAQEAAF4SFAsAAAAQ-file-BgWyEd" [4] Input filter: Removed temporary file: /usr/local/modsecurity/var/tmp/20160723-054813-V5Lo-X8AAQEAAF4SFAsAAAAQ-request_body-pJs7UV
The addition of ModSecurity will change how your web server operates. As with all Apache modules, you pay for the additional flexibility and security ModSecurity gives you with increased CPU and RAM consumption on your server. The exact amount will depend on your configuration of ModSecurity—namely, the rules—and the usage of your server. The following is a detailed list of the various activities that increase resource consumption:
ModSecurity will add to the parsing already performed by Apache, which results in a slight increase in CPU consumption.
The handling of file uploads may require I/O operations. In some cases, inbound data will be duplicated on disk.
The parsing will add to RAM consumption, because every extracted element (e.g., a request parameter) will need to be copied into its own space.
Request bodies and response bodies are usually buffered in order to support reliable blocking.
Every rule in your configuration will use some CPU time (for the operator) and RAM (to transform input data before it can be analyzed).
Some operators used in the rules (e.g., the regular expression operator) are CPU-intensive. Running regular expressions on very large request or response bodies can take a long time—seconds, even.
In practice, this list is important because it keeps you informed; what matters is that you have enough resources to support your ModSecurity needs. If you do, then it doesn’t matter how expensive ModSecurity is. Also, what’s expensive to one person may not be to someone else. If you don’t have enough resources to do everything you want with ModSecurity, you’ll need to monitor the operation of your system and remove some functionality to reduce the resource consumption; virtually everything that ModSecurity does is configurable, so you should have no problems doing so.
It’s generally easier to run ModSecurity in reverse proxy mode, because then you usually have an entire server (with its own CPU and RAM) to play with. In embedded mode, ModSecurity will add to the processing already performed by the web server, so this method is more challenging on a busy server.
For what it’s worth, ModSecurity generally uses the minimal necessary resources to perform the desired functions, so this is really a case of exchanging functionality for speed; if you want to do more, you have to pay more.
The purpose of this section is to map your future ModSecurity activities and help you determine where to go from here. Where you’ll go depends on what you want to achieve and how much time you have to spend. A complete ModSecurity experience, so to speak, consists of the following elements:
This is the basic step that all users must learn how to perform. The next three chapters will teach you how to make ModSecurity operational, performing installation, general configuration, and logging configuration. Once you’re done with those tasks, you need to decide what you want to do with ModSecurity—and that’s what the remainder of the book is for.
Rule writing is an essential skill. You may currently view rules as a tool to detect application security attacks. They are that, but they are also much more. In ModSecurity, you write rules to find out more about HTTP clients (e.g., geolocation and IP address reputation), perform long-term activity tracking (of IP addresses, sessions, and users, for example), implement policy decisions (use available information to make decisions to warn or block), write virtual patches, and even to check on the status of ModSecurity itself.
It’s true that the attack detection rules are in a class of their own, but that’s mostly because in order to write them successfully, you need to know a great deal about application security. For that reason, many ModSecurity users generally focus on using third-party rulesets for attack detection. It’s a legitimate choice. Not everyone has the time and inclination to become an application security expert. Even if you end up not using any inspection rules whatsoever, the ability to write virtual patches is reason enough to use ModSecurity.
The use of existing rulesets is the easiest way to get to the proverbial low-hanging fruit: invest small effort and reap big benefits. Traditionally, the main source of ModSecurity rules has been the CRS project, now hosted with OWASP. On the other hand, if you are keen to get your hands dirty, I can tell you that I draw great pleasure from writing my own rules. It’s a great way to learn about application security. The only drawback is that it requires a large time investment.
ModSecurity is perfectly usable without a remote logging solution and without a GUI (the two usually go together). Significant error messages are copied to Apache’s error log. Complete transactions are usually logged to the audit log. With a notification system in place, you’ll know when something happens and can visit the audit logs to investigate. For example, many installations will divert Apache’s error log to a central logging system (via syslog).
The process does become more difficult with more than one sensor to manage. Furthermore, GUIs make the whole experience of monitoring much more pleasant. For that reason you’ll probably aim to install one of the available remote centralization tools and use its GUI. The available options are listed in the following Resources section.
This section contains a list of assorted ModSecurity resources that can assist you in your work.
The following resources are the bare essentials:
ModSecurity’s web site is probably going to be your main source of information.2 You should visit the web site from time to time, as well as subscribe to receive the updates from the blog.
The official ModSecurity documentation is maintained in a wiki, but copies of it are made for inclusion with every release.3
You’ll want to visit the ModSecurity issue tracker4 for one of two reasons: to report a problem with ModSecurity itself (e.g., when you find a bug) or to check the progress on the next (major or minor) version. Before reporting any problems, go through the support checklist,5 which will help you assemble the information required to help resolve your problem. Providing as much information as possible will help the developers understand and replicate the problem and provide a fix (or a workaround) quickly.
The users’ mailing list (mod-security-users@lists.sourceforge.net) is a general-purpose mailing list through which you can discuss ModSecurity.6 Feel free to ask questions, propose improvements, and discuss ideas. You’ll hear about new ModSecurity versions first through this list.
The CRS project7 is part of OWASP8 and has a separate mailing list (owasp-modsecurity-core-rule-set@lists.owasp.org). Discussions about false positives and the development of new rules also take place in the Core Rules GitHub repository.9
If you’re interested in development work, you’ll need to access the following resources:
The developers’ mailing list is a resource for discussing ModSecurity software development.10 If you do decide to start playing with the source code, use this list to seek advice and to discuss your work. There is also a ModSecurity developers’ guide available with guidelines and examples.11
The source code of ModSecurity is hosted in a GitHub repository, which allows you to access it directly or through a web-based UI.12
Using ModSecurity entirely from the command line is a lot of fun, but reviewing audit and debug logs is difficult without special scripts or higher-level tools. Your best choice for a log centralization and GUI tool is AuditConsole, built by Christian Bockermann.13
AuditConsole is free and provides the following features:
Event centralization from multiple remote ModSecurity installations
Event storage and retrieval
Support for multiple user accounts and support for different views
Event tagging
Event triggers, which are executed in the console
This chapter provided a ModSecurity orientation. I introduced ModSecurity at a high level, discussed what it is and what it isn’t, and what it can and can’t do. I also gave you a taste of what ModSecurity is like and described its common usage scenarios, as well as covered some of the interesting parts of its operation.
The foundation you now have should be enough to help you set off on a journey of ModSecurity exploration. The next chapter discusses installation.
[2] ModSecurity web site (SpiderLabs, retrieved 29 December 2016)
[3] ModSecurity documentation (SpiderLabs, retrieved 29 December 2016)
[4] ModSecurity issue tracker (GitHub, retrieved 29 December 2016)
[5] ModSecurity Support Checklist (SpiderLabs, retrieved 29 December 2016)
[6] ModSecurity Users’ mailing list (SourceForge, retrieved 29 December 2016)
[7] Core Rules Project (OWASP, retrieved 29 December 2016)
[9] CRS GitHub repository (GitHub, retrieved 29 December 2016)
[10] ModSecurity developers’ mailing list (SourceForge, retrieved 29 December 2016)
[11] ModSecurity developers’ guide (SpiderLabs, retrieved 29 December 2016)
[12] ModSecurity source code (GitHub, retrieved 29 December 2016)
[13] AuditConsole (Christian Bockermann, retrieved 15 January 2017)