> ModSecurity Handbook: Getting Started: Preface



I didn’t mean to write this book, I really didn’t. Several months ago I started to work on the second edition of Apache Security, deciding to rewrite the ModSecurity chapter first. A funny thing happened: the ModSecurity chapter kept growing and growing. It hit 40 pages. It hit 80 pages. And then I realized that I was nowhere near the end. That was all the excuse I needed to put Apache Security aside—for the time being—and focus on a ModSecurity book instead.

I admit that I couldn’t be happier, although it was an entirely emotional decision. After spending years working on ModSecurity, I knew it had so much more to offer, yet the documentation wasn’t there to show the way. But it is now, I am thrilled to say. The package is complete: you have an open source tool that is able to compete with the best commercial products out there, and you have the documentation to match.

With this book I am also trying something completely new—continuous writing and publishing. You see, I had published my first book with a major publisher, but I never quite liked the process. It was too slow. You write a book pretty much in isolation, you publish it, and then you never get to update it. I was never happy with that, and that’s why I decided to do things differently this time.

Simply said, ModSecurity Handbook is a living book. Every time I make a change, a new digital version is made available to you. If I improve the book based on your feedback, you get the improvements as soon as I make them. If you prefer a paper book, you can still get it of course, through the usual channels. Although I can’t do anything about updating the paper version of the book, we can narrow the gap slightly by pushing out book updates even between editions. That means that, even when you get the paper version (as most people seem to prefer to), it is never going to be too much behind the digital version.

This book exists to document every single aspect of ModSecurity and to teach you how to use it. It is as simple as that. ModSecurity is a fantastic tool, but it is let down by the poor quality of the documentation. As a result, the adoption is not as good as it could be; application security is difficult on its own and you don’t really want to struggle with poorly documented tools too. I felt a responsibility to write this book and show how ModSecurity can compete with commercial web application firewalls, in spite of being the underdog. Now that the book is finished, I feel I’ve done a proper job with ModSecurity.

If you are interested in application security, you are my target audience. Even if you’re not interested in application security as such, and only want to deal with your particular problems (it’s difficult to find a web application these days that’s without security problems), you are still my target audience.

You don’t need to know anything about ModSecurity to get started. If you just follow the book from the beginning, you will find that every new chapter advances a notch. Even if you are a long-time ModSecurity user, I believe you will benefit from a fresh start. I will let you in on a secret—I have. There’s nothing better for completing one’s knowledge than having to write about a particular topic. I suspect that long-time ModSecurity users will especially like the second half of the book, which discusses many advanced topics and often covers substantial new ground.

But, there is only so much a book can cover. ModSecurity Handbook assumes you already know how to operate the Apache web server. You don’t have to be an expert, but you do need to know how to install, configure, and run it. If you don’t know how to do that already, you should get my first book, Apache Security. I wrote it five years ago, but it’s still remarkably fresh. (Ironically, it is only the ModSecurity chapter in Apache Security that is completely obsolete. But that’s why you have this book.)

On the other end, ModSecurity Handbook will teach you how to use ModSecurity and write good rules, but it won’t teach you application security. In my earlier book, Apache Security, I included a chapter that served as an introduction to application security, but, even then, I was barely able to mention all that I wanted, and the chapter was still the longest chapter in the book. Since then, the application security field has exploded and now you have to read several books and dozens of research papers just to begin to understand it.

Once you go past the first chapter, which is the introduction to the world of ModSecurity, the rest of the book consists of roughly three parts. In the first part, you learn how to install and configure ModSecurity. In the second part, you learn how to write rules. As for the third part, you could say that it contains the advanced stuff—a series of chapters each dedicated to one important aspect of ModSecurity.

At the end of the book is the official reference documentation, reproduced with the permission from Breach Security.

Chapter 1, Introduction, is the foundation of the book. It contains a gentle introduction to ModSecurity, and then explains what it can and cannot do. The main usage scenarios are listed to help you identify where you can use ModSecurity in your environment. The middle of the chapter goes under the hood of ModSecurity to give you an insight into how it works, and finishes with an overview of the key areas you will need to learn in order to deploy it. The end of the chapter lists a series of resources (sites, mailing lists, tools, etc.) that you will find useful in your day-to-day work.

Chapter 2, Installation, teaches you how to install ModSecurity, either compiling from source (using one of the released versions or downloading straight from the development repository), or by using one of the available binary packages, on Unix and Windows alike.

Chapter 3, Configuration, explains how each of the available configuration directives should be used. By the end of the chapter, you get a complete overview of the configuration options and will have a solid default configuration for all your ModSecurity installations.

Chapter 4, Logging, deals with the logging features of ModSecurity. The two main logging facilities explained are the debug log, which is useful in rule writing, and the audit log, which is used to log complete transaction data. Special attention is given to remote logging, which you’ll need to manage multiple sensors, or to use any of the user-friendly tools for alert management. File interception and validation is covered in detail. The chapter ends with an advanced section of logging, which explains how to selectively log traffic, and how to use the sanitation feature to prevent sensitive data from being stored in the logs.

Chapter 5, Rule Language Overview, is the first of the three chapters that deal with rule writing. This chapter contains an overview of the entire rule language, which will get you started as well as give you a feature map to which you can return whenever you need to deal with a new problem.

Chapter 6, Rule Language Tutorial, teaches how to write rules, and how to write them well. It’s a very fun chapter that adopts a gradual approach, introducing the features one by one. By the end of the chapter, you will know everything about writing individual rules.

Chapter 7, Rule Configuration, completes the topic of rule writing. It takes a step back to view the rules as the basic block for policy building. You first learn how to put a few rules together and add them to the configuration, as well as how the rules interact with Apache’s ability to use different configuration contexts for different sites and different locations within sites. The chapter spends a great deal of time making sure you take advantage of the inheritance feature, which helps make ModSecurity configuration much easier to maintain.

Chapter 8, Persistent Storage, is quite possibly the most exciting chapter in the book. It describes the persistent storage mechanism, which enables you to track data and events over time and thus opens up an entire new dimension of ModSecurity. This chapter is also the most practical one in the entire book. It gives you the rules for periodic alerting, brute force attack detection, denial of service attack detection, session and user management, fixing session management weaknesses, and more.

Chapter 9, Practical Rule Writing, is, as the name suggests, a tour through many of the practical activities you will perform in your day-to-day work. The chapter starts by covering whitelisting, virtual patching, IP address reputation and blacklisting. You then learn how to integrate with other Apache modules, with practical examples that show how to perform conditional logging and fix insecure session cookies. Special attention is given to the topic of blocking; several approaches, starting from the simple to the very sophisticated, are presented. A section on regular expressions gets you up to speed with the most important ModSecurity operator. The chapter ends with a discussion of rule sets, discussing how to use the rule sets others have written, as well as how to write your own.

Chapter 10, Performance, covers several performance-related topics. It opens with an overview of where ModSecurity usually spends its time, a list of common configuration mistakes that should be avoided, and a list of approaches that result in better performance. The second part of the chapter describes how to monitor ModSecurity performance in production. The third part tests the publicly available rule sets in order to give you a taste of what they are like, as well as document a methodology you can use to test your own rules. The chapter then moves to rule set benchmarking, which is an essential part of the process of rule writing. The last part of this chapter gives very practical advice on how to use regular expressions and parallel matching, comparing several approaches and explaining when to use them.

Chapter 11, Content Injection, explains how to reach from ModSecurity, which is a server-side tool, right into a user’s browser and continue with the inspection there. This feature makes it possible to detect the attacks that were previously thought to be undetectable by a server-side tool, for example DOM-based cross-site scripting attacks. Content injection also comes in handy if you need to communicate with your users—for example, to tell them that they have been attacked.

Chapter 12, Writing Rules in Lua, discusses a gem of a feature: writing rules using the Lua programming language. The rule language of ModSecurity is easy to use and can get a lot done, but for the really difficult problems you may need the power of a proper programming language. In addition, you can use Lua to react to events, and it is especially useful when integrating with external systems.

Chapter 13, Handling XML, covers the XML capabilities of ModSecurity in detail. You get to learn how to validate XML using either DTDs or XML Schemas, and how to combine XPath expressions with the other features ModSecurity offers to perform both whitelist- and blacklist-based validation. The XML features of ModSecurity have traditionally been poorly documented; here you will find details never covered before. The chapter ends with an XML validation framework you can easily adapt for your needs.

Chapter 14, Extending Rule Language, discusses how you can extend ModSecurity to implement new functionality. It gives several step-by-step examples, explaining how to implement a transformation function, an operator, and a variable. Of course, with ModSecurity being open source, you can extend it directly at any point, but when you use the official APIs, you avoid making a custom version of ModSecurity (which is generally time consuming because it prevents upgrades).

If you purchased this book directly from Feisty Duck, your purchase includes access to newer digital versions of the book. Updates are made automatically after I update the manuscript, which I keep in DocBook format in a Subversion repository. At the moment, there is a script that runs every hour, and rebuilds the book when necessary. Whenever you visit your personal digital download link, you get the most recent version of the book.

I use a dedicated Twitter account (@modsecuritybook) to announce relevant changes I make to the book. By following that account you’ll find out about the improvements pretty much as they happen. You can also follow my personal Twitter account (@ivanristic) or subscribe to my blog, if you are about computer security in general.

In the first two years of its life, I kept ModSecurity Handbook up-to-date with every ModSecurity release. There was a full revision in February 2012, which made the book essentially as good and as current as it was on day of the first release back in 2010. Don’t take my past performance as a guarantee of what is going to happen in the future, however. At the launch in 2010 I offered a guarantee that the book will be kept up-to-date for at least a year from your purchase. I dropped that promise at the end of 2011, because I could see the possibility that I would stop with the updates at some point. I will keep my promise until the end of 2012, but I don’t know what will happen after that.

To get in touch with me please write to ivanr@webkreator.com. I would like to hear from you very much, because I believe that a book can fulfill its potential only through the interaction among its author(s) and the readers. Your feedback is particularly important when a book is continuously updated, like this one is. When I change the book as a result of your feedback, all the changes are immediately delivered back to you. There is no more waiting for years to see the improvements!

Ivan Ristić is a respected security expert and author, known especially for his contribution to the web application firewall field and the development of ModSecurity, the open source web application firewall. He is also the author of Apache Security, a comprehensive security guide for the Apache web server. A frequent speaker at computer security conferences, Ivan is an active participant in the application security community, a member of the Open Web Application Security Project (OWASP), and an officer of the Web Application Security Consortium (WASC).

Brian Rectanus is a developer turned manager in the web application security field. He has worked in the past on various security software related projects such as the IronBee open source WAF framework, the ModSecurity open source WAF and the Suricata open source IDS/IPS. Brian is an open source advocate and proud `NIX loving, Mac using, non-Windows user who has been writing code on various `NIX platforms with vi since 1993. Today he still does all his development work in the more modern vim editor—like there is any other—and loves every bit of it. Brian has spent the majority of his career working with web technology from various perspectives, be it manager, developer, administrator or security assessor. Brian has held many certifications in the past, including GCIA and GCIH certification from the SANS Institute and a BS in computer science from Kansas State University.

To begin with, I would like to thank the entire ModSecurity community for their support, and especially all of you who used ModSecurity and sent me your feedback. ModSecurity wouldn’t be what it is without you. Developing and supporting ModSecurity was a remarkable experience; I hope you enjoy using it as much as I enjoyed developing it.

I would also like to thank my former colleagues from Breach Security, who gave me a warm welcome, even though I joined them pretty late in the game. I regret that, due to my geographic location, I didn’t spend more time working with you. I would especially like to thank—in no particular order—Brian Rectanus, Ryan Barnett, Ofer Shezaf, and Avi Aminov, who worked with me on the ModSecurity team. Brian was also kind to work with me on the book as a technical reviewer, and I owe special thanks to him for ensuring I didn’t make too many mistakes.

I mustn’t forget my copyeditor, Nancy Kotary, who was a pleasure to work with, despite having to deal with DocBook and Subversion, none of which is in the standard copyediting repertoire.

For some reason unknown to me, my dear wife Jelena continues to tolerate my long working hours. Probably because I keep promising to work less, even though that never seems to happen. To her I can only offer my undying love and gratitude for accepting me for who I am. My daughter Iva, who’s four, is too young to understand what she means to me, but that’s all right—I have the patience to wait for another 20 years or so. She is the other sunshine in my life.