CoreDNS memory exhaustion vulnerability in custom transports

CoreDNS, as detailed in a GitHub Security Advisory, has a high-severity vulnerability affecting its DNS-over-HTTPS (DoH and DoH3), DNS-over-QUIC (DoQ), and DNS-over-gRPC listeners. The issue arises from how CoreDNS parses attacker-controlled DNS section counts before validating them, allowing unauthenticated clients to exploit DNS name compression. A single 65,533-byte request can allocate over 10 MiB of memory during unpacking, and concurrent requests can lead to memory exhaustion, potentially terminating the CoreDNS service.

The vulnerability stems from the affected request paths directly invoking `dns.Msg.Unpack` without applying early validation policies, unlike the miekg/dns UDP and TCP server, which uses `DefaultMsgAcceptFunc` to limit section counts before unpacking. This bypass in CoreDNS's custom transports means plugin-level rate limiting or request handling cannot mitigate the issue. The advisory recommends applying `dns.DefaultMsgAcceptFunc` to the fixed header before calling `Msg.Unpack` to address the flaw.

Operators should evaluate whether their CoreDNS deployment uses the impacted custom transports and assess memory usage patterns under load. Systems with limited memory or those exposed to public queries may be particularly vulnerable to exploitation. Before upgrading or applying fixes, ensure compatibility with existing configurations and validate whether the updated transport logic aligns with your operational requirements.