Introduction
open-appsec is an open-source WAF solution that provides automatic, preemptive, machine-learning-based threat prevention for web applications and web APIs. It does not require any traditional threat signatures and reduces administrative effort significantly. Deployment of open-appsec is supported on Docker, Kubernetes and Linux, integrating with many popular reverse proxy solutions like NGINX, Kong API Gateway, APISIX API Gateway, NGINX Proxy Manager, Docker SWAG, Envoy (soon) and Istio (soon).
In this blog, we are announcing the (beta) release of a new docker-compose-based deployment option in addition to the existing option to deploy on Docker with docker run command. You will find the deployment steps in this blog.
This new deployment option also introduces various new capabilities and benefits for the deployment and management of open-appsec, which are explained below.
The new docker-compose.yaml files are now available for all supported open-appsec integrations, you can find the full documentation and download links in the docs!
Benefits and new functionality introduced with the new docker-compose-based deployment:
General benefits of new docker-compose-based deployment:
Using docker-compose for deployment is ideal for DevOps/DevSecOps scenarios and processes like GitOps CD, as it is a fully declarative deployment type.
Simplified deployment using a single file for deploying all containers relevant for open-appsec.
The .env file provides the most relevant configuration options in a separate, easy-to-edit text file, e.g. for providing the token for connecting to the central management WebUI (SaaS).
New functionality for standalone, local-only deployments:
In the case of deploying the new docker-compose.yaml locally managed, in standalone environments that have no connection to the open-appsec central management WebUI (https://my.openappsec.io), the following functionality is now available also directly on CLI. Using the newly released "open-appsec-tuning-tool" CLI application you now can:
View open-appsec agent statistics locally on CLI e.g.:
Total request amount
Critical and High request amount
Unique URLs and Unique Source
Legitimate requests
Malicious requests
and more…
View the learning status of the contextual machine learning model on CLI:
E.g. Kindergarten, Primary School, High School, …
Receive configuration recommendations based on the current learning level on CLI
Receive and manage tuning suggestions relevant to supervised learning on CLI
Note: When you are connected to the central WebUI, using the open-appsec-tuning-tool is neither required nor supported, as in this case all of the above can be done comfortably and centrally from the open-appsec WebUI interface.
Introducing new containers in the new docker-compose.yaml (for standalone deployments only):
To provide the above new functionality for local, standalone deployments (only! in this case), the new docker-compose.yaml file will deploy a few more containers in addition to the open-appsec agent and (in this case) nginx-attachment (NGINX with open-appsec module) containers, here’s what they do in short:
smartsync: responsible for syncing the learning in standalone deployments between appsec transaction handlers in the same agent and between multiple agents.
smartsync-shared-files: provides shared storage which is used by agent, smartsync and smartsync-tuning containers for information exchange.
smartsync-tuning: provides tuning suggestions to the user based on security events reported by the agent containers.
postgres: postgres-database used by the smartsync-tuning container.
The source code for all of these containers is open-source and available on GitHub:
open-appsec WAF source code repository: https://github.com/openappsec
Postgres source code repository: https://github.com/postgres
Benefits for Testing, Training, and Demo – includes Juice Shop deployment option:
The new docker-compose-based deployment option contains also an optional example deployment of a juiceshop container (OWASP Juice Shop | OWASP Foundation), the relevant configuration required for each specific proxy to expose the Juice Shop Web App is provided for download as well. This allows easy testing of open-appsec WAF with an actual highly vulnerable Web Application available as an exposed backend.
For further instructions on how to deploy with juice-shop please see the comments in the .env file (provided alongside each docker-compose.yaml) file or https://docs.openappsec.io.
How to deploy open-appsec with docker-compose tool
Note: This example is for docker-compose-based deployment of open-appsec integrated with NGINX, docker-compose.yaml files for open-appsec integrated with Kong, APISIX, NGINX Proxy Manager and SWAG are available as well in a very similar way, please see docs for detailed instructions.
Prerequisites:
Linux Docker Host with root permission
docker-compose tool installed
(Optional, Recommended) Sign-Up and Login to open-appsec WebUI Portal
If you want to centrally manage your open-appsec WAF deployment via the open-appsec WebUI (SaaS) or if you want to locally manage your open-appsec WAF deployment but still connect to central WebUI for viewing the local configuration (in read-only), central monitoring, logging, and reporting, follow the instructions below to sign up and login to the WebUI my.openappsec.io.
(Optional, Recommended) Create a deployment profile for the open-appsec deployment in WebUI Portal
If you signed-up and logged in to the WebUI Portal (see prerequisite above), now follow the instructions below to create a new deployment profile for your open-appsec deployment. Once done, don't forget to copy the profile token after policy installation as this is needed in the installation steps further below.
Deployment
Create a folder for your new open-appsec deployment and switch to that folder, e.g.
mkdir open-appsec-deployment
cd ./open-appsec-deploymentDownload the docker-compose.yaml file for your desired open-appsec integration.
The following docker-compose.yaml file is for the open-appsec integration with NGINX:
wget https://raw.githubusercontent.com/openappsec/openappsec/main/deployment/nginx/docker-compose.yamlDownload the .env file for your desired open-appsec integration and adjust the configuration to your requirements as described below:
The following .env file is for the open-appsec integration with NGINX:
If you created a deployment profile in the WebUI and copied the Token from it:
Edit the .env file and add your token to the key APPSEC_AGENT_TOKEN.
If you did not create a deployment profile in the WebUI and do not want to connect your deployment to central WebUI (SaaS) at all:
Edit the .env file and add standalone to the COMPOSE_PROFILES key, which will set the standalone mode and activate the deployment of additional containers which are required only when not connected to the WebUI at all (resulting in standalone, locally, declaratively managed deployment).
Replace user@email.com in the .env file with your own email. (More details below.)
If you have decided to manage your open-appsec deployment locally and declaratively with local_policy.yaml file:
Download the initial declarative configuration file for open-appsec into new subfolder ./appsec-localconfig:
mkdir ./appsec-localconfig
wget https://raw.githubusercontent.com/openappsec/openappsec/main/config/linux/v1beta1/prevent/local_policy.yaml -O ./appsec-localconfig/local_policy.yaml(This example local configuration file is already set to mode: prevent-learn so that open-appsec will prevent attacks right from the start.)
7. Perform the deployment
docker-compose up -d
Congratulations, you successfully deployed open-appsec WAF!
Recommended next steps after deployment
If you connected to the central WebUI AND configured your deployment profile in the WebUI to "This management" mode for centrally managing open-appsec configuration:
Create one or more assets in the WebUI that represent web applications and/or Web APIs that you want to be protected by open-appsec WAF and allow you to adjust the open-appsec configuration specifically for each of them.
Make sure to link your assets to the specific WebUI Profile that you created earlier (General -> Profiles) and adjust the Threat Prevention mode to Detect-Learn or Prevent (Threat Prevention -> Mode), the steps are described here:
If you decided to locally, declaratively manage open-appsec (with or without connection to central WebUI in "Declarative configuration" mode):
Follow the steps described here to configure your open-appsec deployment using the local_policy.yaml file: Configuration Using Local Policy File (Docker) In case you connected your locally managed deployment also to the central WebUI in "Declarative Configuration" mode, you can check security logs and view agent status and configuration also in the central WebUI at https://my.openappsec.io.
How to deploy and use the new open-appsec-tuning-tool
open-appsec WAF supports both, local declarative configuration as well as centrally managing the configuration from the open-appsec WebUI (SaaS).
open-appsec is based on a contextual machine learning engine, which allows preemptive prevention of unknown and known attacks without any signatures. open-appsec’s online machine learning model is constantly learning and improving based on the observed and inspected traffic. It also allows “supervised learning”, meaning that the user can review learning suggestions presented by open-appsec and confirm or reject those, to further improve the accuracy of the ML model. (There’s also a second ML model, which is trained offline with millions of malicious and benign requests).
Based on the current learning progress, which requires a certain amount of traffic from various sources to be observed, open-appsec suggests the ideal enforcement configuration (aligned with the current learning level).
Specifically for usage in local, declarative configuration management scenarios, a new open-appsec-tuning-tool application was released, which provides access to e.g. machine learning and tuning-related functionality now also on CLI level, which until now was only accessible in the central WebUI.
Note: In case you are connected to the central WebUI the usage of the open-appsec-tuning-tool is neither required nor supported as in this case all of the above can be done comfortably and centrally from the open-appsec WebUI interface.
Prerequisites for using the open-appsec-tuning-tool:
Existing open-appsec deployment (Docker or Kubernetes)
If your deployment is on Docker: To have access to the below functionality your deployment must have been done using the new "docker-compose" based deployment option as described in this blog, as only this deployment option will deploy the required additional containers for local learning and tuning when configured correctly for standalone deployment. See here for details: Deploy With Docker-Compose
No Agent Token configured in the deployment (no connection to central WebUI)
Make sure that the agent already received some traffic already, as otherwise the open-appsec-tuning-tool will not be able to provide any statistics, recommendations, etc.
Download and installation of the open-appsec-tuning-tool:
1. Download the open-appsec-tuning-tool
2. Make the open-appsec-local-tuning tool executable
chmod +x ./open-appsec-tuning-toolUsing the open-appsec-tuning-tool
Run the open-appsec-tuning-tool to get an overview of the available options:
./open-appsec-tuning-tool
Select among the available options presented:
View statistics
Select [1] to view current learning statistics, learning progress and receive recommendations for configuration based on those.
Manage tuning suggestions for learning
Select [2] to view tuning suggestions in case there are some available based on observed traffic and learning state.
To perform tuning:
- First select a tuning suggestion based on its ID.
- Review the relevant logs presented which allow you to better decide what decision to take for that suggestion. (You also have the option to export those logs into a .csv file.)
- Take a decision on that tuning suggestion by setting it to "malicious" or "benign".
View tuning decisions
Select [3] to view tuning decisions which you already took based on earlier tuning suggestions.
Full documentation for the open-appsec-tuning-tool is available here:
More information about open-appsec is available here:
Website: https://www.openappsec.io
To learn more about how open-appsec works, see this White Paper and the in-depth Video Tutorial. You can also experiment with deployment in the free Playground.
