【Table of contents】
- Introduction
- What is Text Transformation?
- List of AWS WAF Text Transformation Features
- Usage
- Use with WafCharm
- Conclusion
1. Introduction
On June 25, 2021, the number of transformation patterns for AWS WAF Text Transformation has increased significantly. In this article we’ll address what Text Transformation is and how it can be used.
2. What is Text Transformation?
Text Transformation is a function that pre-processes strings before they are inspected by WAF. (For example, the delimiter of directory path is "/" in Linux, while it is "\" in Windows.) The same signature can be used to check for input patterns such as Base64-encoded input and non-Base64-encoded input, and can be used for both input patterns.
Once the expressions are unified, there is no need to prepare multiple check signatures according to the expressions.
3. List of AWS WAF Text Transformation Features
https://docs.aws.amazon.com/waf/latest/APIReference/API_TextTransformation.html
From a total of 21 types of transformation rules, up to 10, (as of June 2021,) can be applied simultaneously. However, since all of the described transformation processes are performed in order, depending on the decoding process, it may be necessary to separate the conditions based on or conditions, etc.
| Type | New? | Description |
|---|---|---|
| BASE64_DECODE | New | Decode base64 |
| BASE64_DECODE_EXT | New | Decode tolerant base64, such as ignoring invalid characters |
| CMD_LINE | Normalize the input pattern used on the command line | |
| COMPRESS_WHITE_SPACE | Normalize whitespace | |
| CSS_DECODE | New | Decode strings encoded with CSS 2.x escape rule |
| ESCAPE_SEQ_DECODE | New | Decode ANSI C escape sequences |
| HEX_DECODE | New | Normalize hexadecimal representation to binary representation |
| HTML_ENTITY_DECODE | Decode html character entity reference | |
| JS_DECODE | New | Decode Javascript escape sequences |
| LOWERCASE | Convert uppercase to lowercase | |
| MD5 | New | Hash with md5 |
| NONE | Do nothing. It is used when Text Transformation is not used. | |
| NORMALIZE_PATH | New | Normalize special directory representations such as “//” and “../” |
| NORMALIZE_PATH_WIN | New | Convert Windows backslash representation to slash and then equivalent to NORMALIZE_PATH |
| REMOVE_NULLS | New | Remove nulls |
| REPLACE_COMMENTS | New | Replace the C comment “/ ... /” with a space. |
| REPLACE_NULLS | New | Replace NULL with blank |
| SQL_HEX_DECODE | New | Decode the hexadecimal representation of SQL |
| URL_DECODE | Decode URI-encoded representations | |
| URL_DECODE_UNI | New | Similar to URL_DECODE, but supports Microsoft% u encoding |
| UTF8_TO_UNICODE | New | Convert UTF-8 to Unicode |
4. Usage
The originally provided URI_DECODE and LOWERCASE can be used in a very general way. Even in the creation of simple signatures, there will be many opportunities to consider URI encoding and case.
Among the new additions, NORMALIZE_PATH can also be used in many cases.
The others seem to be for power users who want to supplement their own special APIs that are falsely detected by commercial rules.
Unless you are a WAF specialist, there may be a situation where you can use a dedicated whitelist rule that only allows certain inputs to pass.
5. Use in conjunction with WafCharm
It is not expected that WafCharm users will find WafCharm more useful by using Text Transformation together, but there is a possibility that WafCharm rules will be optimized using Text Transformation, or used for rule customization by WafCharm support in a future update.
6. Conclusion
The Text Transformation pattern added this time seems to have been imported from the advanced Text Transformation feature of ModSecurity (https://github.com/SpiderLabs/ModSecurity), an open source WAF.
This feature is designed for more advanced users, as it requires greater technical literacy to operate effectively.
If you are a WafCharm user, you won’t need to utilize it. You can leave the customization and operation of signatures to us as usual.
