Overview
In the field of web application and API security, schema is a vital concept that helps developers to ensure that their applications are secure, reliable, and resilient against attacks. A schema is essentially a blueprint or structure that defines the format and content of the data that is exchanged between the client and the server.
Schemas can be used for various security-related purposes such as input validation, output encoding, ensuring correct implementation of security protocols, and providing clear documentation. By defining a schema for the data exchanged between the client and the server, developers can ensure that the data is in the expected format and that it is free from malicious content. This can help prevent attacks such as SQL injection, cross-site scripting (XSS), and other injection-based attacks.
In the context of APIs, schema can be used to define the expected request and response formats, the required data types, and the allowed values for each field. This can help prevent common API security issues such as data tampering, data leakage, and unauthorized access. Schemas can also be used to enforce access control policies by specifying which users or roles are allowed to access certain parts of the application or API. This can help prevent unauthorized access and protect sensitive data.
In this blog we describe how we used the open-appsec engine’s Schema Validation capability to protect our own APIs. We also used our own product to validate the exact schema of the APIs we accept from our agents.
The open-appsec engine provides numerous security capabilities for web applications and web APIs. While there are many automatic security engines that detect and prevent potential attacks, you must first:
Define the exact REST API that is accepted by the web server.
Provide the definition, in OpenAPI format, to the open-appsec engine so it can validate the requests accordingly.
While some attacks and anomalies can be detected without any administrator input, if you provide an API specification which defines, for example, that a certain key can accept only integer values lower than 10, you can elevate the security level.
Check Point’s cloud systems, which supply the backend services for the open-appsec engine itself, also accept APIs from various agents. We protect our cloud systems at the highest level possible. Among the elements providing this security is our own product, an open-appsec Gateway.
Preparation for Schema Validation configuration
Use Case:
An existing system that accepts and protects APIs to web servers in the cloud, using open-appsec. We requested to add schema validation.
Prerequisites:
Assign a schema owner if one does not already exist and specify that all new schema changes must go through the owner.
The new schema owner must work with all API owners to create a unified openAPI YAML schema file of our current version (if not previously available) and name that file ‘v1.0’ (as this was our version number). Using a version in the file name is very important. When you have a running system that was not validated by an external schema validation engine such as open-appsec, mistakes can occur on either the server side or the client side and the version might change due to the validation process, in which case new version files will be created. For this particular reason it is important to implement this security engine – i.e. to externally verify, as security officers, that there are no mistakes.
Verify there is a single source of truth to the schema and all changes to it are reviewed if this did not exist previously.
Ownership of a Server vs. Ownership of a Server/Client
There are 2 optional use-cases:
We protect our server, but we are not the owners of the clients that connect to it. This is the case in most web servers used by web browsers, and any service that provides API to its users.
We own the servers and clients. This was the case for us, as we are also the owners of the agents and their code that connects to the cloud backend.
Applying Schema Validation in Detect Mode
When you apply schema validation to an existing live system, iterations are almost always involved.
In the first iteration:
1. We browsed to the web asset via ‘Cloud’ > ‘Assets’.
2. We edited the Asset and browsed to ‘Threat Prevention’ > ‘Schema Validation’ as explained in the open-appsec documentation.
3. We made sure that the sub-practice of Schema Validation is active in “Detect” mode and uploaded the openAPI YAML file we created in the preparation phase. 4. We then enforced the policy and let the system run for approximately 24 hours.
These scenarios happened in each iteration:
We browsed to view all schema validation logs under ‘Monitor’ > ‘All Events’ and filtered the AppSec ‘Incident Type’ to show only “Schema Validation” Logs.
For each log, we scrolled through the log card to view the Transaction and Threat Prevention details:
The main fields of interest are:
In the Transaction section, HTTP URI path and HTTP method explains the exact URI in the request that was inspected in this event.
In the Threat Prevention section, Match Reason provides a clear text explanation for the reason the log was created.
In the Threat Prevention section, Matched Parameter provides the location within the request where the request did not match the schema, and when applicable, Matched Sample provides the value that did not match the schema.
Each log is a potential action item. The logs must be analyzed to determine which of these is the reason the log was created:
The schema lacks a definition of API that is supported by the server. Action item – Fix the schema and work with the relevant API owner to see if additional components are missing from the schema definition.
An actual bad request which should be prevented and does not get a functional reply from the server. Action item – Usually there is no action item in this case. The exception is our use-case, where we are the owners of the client code (agents connecting to the cloud). In this case, this usually indicates a bug in the client code.
An older API or option within an API is still in use by clients yet should be considered deprecated by the server. In other words, this API receives a functional answer from the server, but we no longer wish to support it. Action item – You must consult with the API owner. The server replies a functional answer to this API, meaning it was not yet deprecated. Were the clients informed this API should be deprecated? Should the API continue to be active, at least temporarily? Adjust the openAPI schema file according to the answer. If necessary, add the API, or API options, so it will not be deprecated abruptly.
Filter out the unique part of the log category. For example, filter out the HTTP URI to look for additional logs that may create additional action items.
If modifications to the schema files are required, save a new file with a higher version, re-upload it and enforce policy again. If server-side fixes are needed, wait for R&D to confirm they are implemented. Then perform another iteration.
Configuring Schema Validation in Prevent Mode
While it is important to continue looking at the log status on a periodic basis even after the configuration described in this blog, at some point in the iteration all the current logs fall under case #2 above, meaning malicious requests that we expect to be blocked by the Schema Validation engine.
It is also important to verify that a correct process is defined:
API owners on the R&D side must notify the selected schema owner on the security administration team. This is important both for review of the schema changes, and for proactively modifying the schema file used by open-appsec for schema validation.
As best practice, a staging environment should exist. In such a staging environment you can upload the new schema file when the new/changed API is deployed for test purposes. This should be done before changing the configuration in the open-appsec gateway or agent that protects production environments.
Further information about open-appsec engine’s Schema Validation capability, can be found in our documentation.
To learn more about open-appsec, visit our hands-on lab or our additional articles.
