Scaling Security Insights: how we achieved a 10x increase in global scanning capacity

Summary

Cloudflare Security Insights system now processes over 120 scans per second, providing frequent insights for all customers. By optimizing Kafka consumers, Postgres queries, and our API, we scaled our throughput 10x without adding hardware.