Waf Charm

Blog

AWS WAF

How to get AWS WAF Sample Logs and Full Logging

Overview

AWS WAF supports Sample Logs and Full Logging.
Chapter 1 is for how to get Sample Logs, Chapter 2 is for how to output Full Logging.

Contents

  1. 1. How to get AWS WAF Sample requests (Sampled Logs)
  2. 2. How to output AWS WAF Full Logging to S3 via Kinesis Firehose (Full Logging)

1. How to get AWS WAF Sample requests (Sampled Logs)

Choose "Go to AWS WAF".

 

Choose "Web ACLs".

Go to the “Requests” tab to get Sample Logs.
Click to the mark ▶︎ right next to a sample log, and then you can see the data.

2. How to output AWS WAF Full Logging to S3 via Kinesis Firehose (Full Logging)

What is Kinesis Firehose
Kinesis Firehose is a service that saves data generated in near real time to the set output destination. You can save the generated data in S3 or Redshift and analyze it. First, create a Kinesis Firehose delivery stream, then link it to AWS WAF.

2-1. Setting Kinesis Firehose

Go to Amazon Kinesis page.

 

Choose “Create delivery stream”
In case of ALB, keep "Region" same as WAF (Web ACL), however with CloudFront, set "Region" to "N. Virginia (Eastern US)".

 

Enter "Delivery stream name* ". You can use whatever you like, but please add "aws-waf-logs-" at the beginning.

For "Source* ", select "Direct PUT or other sources"

 

Choose “Next”.

 

Since we want to output log data from AWS WAF to S3 in its original form, please select "Disabled" for "Record Transformation* " in "Transform source records with AWS Lambda".

 

Since we need to output record to S3 as it is, select "Disabled" for "Record format conversion* " in "Convert record format" and choose "Next".

 

For "Destination", select "Amazon S3".

 

Choose an existing bucket, or choose New S3 Bucket. If you create a new bucket, type a name for the bucket and choose the region your console is currently using. If necessary, set Prefix for S3 bucket and insert "/" after Prefix. Choose "Next".

 

Decide the size and time to buffer the data. We recommend 60 seconds.

 

Next, in "S3 compression and encryption", to compress the log, select "GZIP" in "S3 compression" to minimize the capacity of S3.
For "S3 encryption", select "Disabled".
"Error logging" is "Enabled" as default, so keep it that way.

 

Choose “Create new or choose”

 

In the transition page, select an "IAM Role" or create new.
“Role Name” can be anything.
And then choose “Allow”.

After confirming that the role created earlier is set in "IAM role", choose "Next".

 

Finally, in the review, make sure that there are no mistakes in the configuration.

 

If there is no problem in the review, proceed by choosing "Create delivery stream".

 

Wait, while it's been created.

 

Delivery system created.

 

 

2-2. Linking Kinesis Firehose to AWS WAF

From here on, we will link Kinesis Firehose to AWS WAF.
By doing this, it is possible to save the request log that until now was able to acquire only part of log data up to 3 hours ago to S3.

 

Select “WAF & Shield” from the service.

 

Choose "Go to AWS WAF".

 

Choose "Web ACLs" from the side bar.

 

Choose the applicable Web ACL.
Go to the “Logging” tab.
And then choose “Enable Logging”.

Select "Kinesis Firehose" created earlier in "Amazon Kinesis Data Firehose". "Redacted fields" allows you to select fields that is not an output in the log. For example: URI, Query string
When you are done, choose "Create".

Set up is done.

 

You can now output AWS WAF Full Logging to S3 via Kinesis Firehose. (Example below)

 

Below is a comparison between Sample Logs and Full Logging.

Parts visible in Sample Logs Parts visible in Full Logging
Source IP Client IP
URI URI
Matches rule Rule ID ※1
Action Action
Time Timestamp
Country Country
Method httpMethod
Host Host
Content-Length Content-Length
upgrade-insecure-requests upgrade-insecure-requests
user-agent user-agent
accsept accsept
accept-encoding accept-encoding
accept-language accept-language
cookie cookie
format version
Web ACL ID
RuleType
http Source Name
http Source Id
rule Group List
rateBased Rule List
non Terminating Matching Rules
args
HTTP Version
Request ID ※2

※1 The value will change when the rule is updated.
※2 The value becomes null in ALB (Application Load Balancer).