Can't Properly Export Firewall logs. Erroneous reporting on the Dashboard

Can't Properly Export Firewall logs. Erroneous reporting on the Dashboard

Created By: Florence Marc Agdon



Date Happened: November 21, 2021



Components: Cloudflare WAF



Platform: ALL



Root Cause: 

  1. Firewall events visibility is skewed on the Dashboard and general difficulties on exporting logs from the Module because of the solution's limitations


Findings:

  1. Total number of CF's WAF firewall event reporting is anomalous since the data displayed is sampled and the displayed result also depends on numerous factors like the Managed OWASP Ruleset among others. It is aknown limitation of Cloudflare as stated on their documentation: https://developers.cloudflare.com/analytics/graphql-api/limits Additionally: We can see the actual GraphQL API calls from the browser developer tool and use it as a point of reference so that we could query on the GraphiQL to fetch the required logs or data. In their own words: "You could explain this to your customer, otherwise, if they want the full firewall logs, our Logpush would be great to achieve it https://developers.cloudflare.com/logs/about." "Because sampling is primarily adaptive and automatically adjusts to provide an accurate estimate, the sampling rate cannot be directly controlled. Enterprise customers have access to raw data via Cloudflare Logs..." a separate component of cf that needs requires a dedicated web server.


Resolution:
  1. As a workaround, Cloudflare recommends the following;
  2. 1. Utilize the Cloudflare API ( https://api.cloudflare.com/
    1. 1.2. Fetch the required logs by using GraphiQL software by querying a valid request on the left side of the inteface.
    2. 1.3. Copy and paste the output texts on your favorite IDE/text-editor. Trim the trailing texts from the beginning up unto the second ''['', and trim the trailing texts from the end of the passage up until the second seen "]". This process would make the passage a valid JSON file. Save the trimmed text into a convenient directory.
    3. ​1.4. Open your Excel Workbook where you would want to compile your logs and import the saved file from step 1.3 into the worksheet. Now you have the most accurate, aggretated data from a particular timerange you would want to fetch in the dashboard. Cloudflare recommends to us to utilize their logpull functionalities and/or use LogPush before confirming that the mentioned workaround above, is correct.

    • Related Articles

    • Cloudflare Errors

      https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/

    • 5xx Errors

      https://developers.cloudflare.com/support/troubleshooting/cloudflare-errors/troubleshooting-cloudflare-5xx-errors/

    • Site unaccessible

      Resolution: Check first if the site is accesible via direct/internal access. If not, the issue is not on Cloudflare

    • JS File are not parsed

      Created by: Elia Date Happened: May 7, 2020 Component: CxSAST Platform: Windows Root Cause:  No Error Message - Upon checking the Scan Engine logs you can see that JS files are not being parsed. Results in the report might appear to be too few. ...

    • ActiveMQ Error

      Created by: N/A Date Happened: August 5, 2020 Component: CxSAST Platform: Windows Root Cause: ActiveMQ Error Findings: Check the activeMQ logs. You will see that the ActiveMQ hostname is not being resolved. Resolution: Update the CxDB components ...