SPF "too many DNS lookups": why it happens and how to fix it

What the error means

Checking SPF is not a single DNS query. The receiver fetches your record, and many of the terms in it send it off to fetch more: another domain's SPF record for each include, address records for a, mail servers for mx. To keep a single message from triggering an unbounded chain of queries, RFC 7208 caps the number of those terms at 10 per evaluation. A receiver that reaches the cap must stop and return permerror, the result the RFC defines as records that could not be correctly interpreted and that need the DNS operator to fix them.

Two details make this error easy to underestimate. The first is that receivers evaluate the record from left to right and stop at the first match. A server listed in an early ip4 term, or in the first include, can still get a pass, because the check ends before the limit is hit. A message whose check has to go further, such as mail from the service listed last or from a forger who matches nothing, runs into the cap and gets the error. So the failure is partial and depends on the order of your record, which is why it often goes unnoticed for weeks.

The second is what happens next. A receiver that rejects mail because of a permerror should answer with SMTP code 550 and the enhanced status 5.5.2. Many receivers do not reject on SPF alone and pass the result on to DMARC instead. DMARC only counts an SPF pass, so a permerror simply removes SPF from the decision. If the message also carries a valid DKIM signature aligned with your domain, DMARC still passes. If it does not, DMARC fails, and with a quarantine or reject policy the message goes to spam or is refused.

What counts as a lookup

Section 4.6.4 of the RFC names six terms that cost one lookup each time they are evaluated:

• include fetches another domain's SPF record, and everything that record triggers counts as well.
• a looks up the A or AAAA records of a host name.
• mx looks up a domain's MX records, then the addresses of those mail servers.
• ptr looks up the reverse DNS of the connecting IP; the RFC says it should not be published at all.
• exists checks whether a constructed name resolves, often built from macros.
• redirect= hands the whole evaluation to another domain's record.

ip4, ip6 and all cost nothing, because the receiver can compare the connecting address without asking DNS. The exp= modifier is not counted either; it is only fetched later, to explain a failure. That asymmetry is the key to most fixes: an address written out in the record is free, while a name that has to be resolved is not.

The count is global. The RFC requires a single limit across the whole evaluation, so a lookup inside an included record counts against your 10 exactly like one in your own record. One include that looks innocent in your DNS panel can cost four or five lookups once the provider's own includes are followed.

Two smaller limits sit on top. An mx term must not need more than 10 address lookups for the mail servers it finds; past that, the term itself returns a permerror. And the RFC asks receivers to allow no more than 2 void lookups, meaning queries that come back with no records or with a "name does not exist" answer. A third one is a permerror too. Typical causes are a: or mx: entries for hosts that were retired years ago and includes for services that removed their SPF record.

Counting a real-looking record

Here is the kind of record that ends up over the limit without anyone noticing. The domain uses a mailbox provider, a CRM and a newsletter tool, and someone added mx and a along the way. The names are examples, but the shape is typical:

v=spf1 mx a include:_spf.mailhost.example include:spf.crm.example include:send.news.example ~all

• mx and a at the top: 2 lookups.
• include:_spf.mailhost.example: 1 lookup. Its record contains three further includes, one per network block: 3 more. Running total: 6.
• include:spf.crm.example: 1 lookup. Its record has one include of its own and the rest as ip4 ranges: 1 more. Running total: 8.
• include:send.news.example: 1 lookup. Its record includes two regional records: 2 more. Running total: 11.

The receiver hits the cap inside the newsletter tool's include. Mail from the mailbox provider and the CRM still passes, because those terms come first; newsletters, and every forger, get a permerror. Nothing in the record itself looks wrong, which is the trap: the count lives in other people's DNS and changes whenever a provider reorganizes its records.

That is why the only reliable count is a live one. The SPF checker follows every include, a, mx and redirect the way a receiver would, shows the whole include tree, and puts the total next to the limit as "DNS lookups: N / 10", with the number of empty lookups beside it. Terms that rely on macros cannot be expanded without a real message, so it tells you when it skipped one. The domain health check marks a record that needs 9 or more lookups as a warning even while it is within the limit, because one more include from a provider would tip it over. For the basics of how the record is put together, see What is SPF?

Fixes, safest first

Work down this list and re-count after each change. Only step 4 adds something to maintain: an address you write out has to be updated by hand if that server is ever renumbered.

1. Remove services you no longer use. Old records collect includes for tools that were cancelled long ago. Each one still costs lookups, and an include for a domain that has since dropped its SPF record also adds a void lookup. Ask whoever owns the tool before deleting, then delete.
2. Drop includes the service does not need. SPF checks the domain in the return-path, not the one in the From line. Many sending services use their own return-path or a subdomain of yours, and then an include on your root record does nothing. Mailchimp uses its own return-path and authenticates through DKIM, and Brevo authenticates through DKIM and needs no SPF entry. Postmark's custom return-path is a CNAME on a subdomain and needs no change to your root record, and SendGrid only needs include:sendgrid.net if you turned its automated security off. The setup guides say for each service whether it belongs in your record.
3. Move a sender to its own subdomain. A service that sends with a return-path such as mg.example.com is checked against the SPF record of that subdomain, which has its own budget of 10. Mailgun is usually set up this way, and Amazon SES does the same with a custom MAIL FROM domain. Under relaxed DMARC alignment the subdomain still counts as your domain, so DMARC keeps passing.
4. Replace a and mx with addresses you control. If the servers behind them are your own and their addresses rarely change, write them out as ip4 or ip6. RFC 7208 describes the trade-off directly: a makes renumbering easy at the cost of a lookup for every receiver, and unless such changes are common, the cheaper mechanisms are better. Remove mx entirely if the servers that receive your mail are not the ones that send it, for example a hosted mailbox whose provider you already authorize through its include.
5. Delete ptr. The RFC says it should not be published. It is slow, depends on DNS that the connecting party controls, and still costs a lookup.

After those steps, a domain with a mailbox provider, one transactional service and its own server often looks like this, with the transactional mail moved to a subdomain that carries its own record:

Flattening, and why it is a last resort

Flattening means replacing an include with the IP ranges it resolves to today. Ten includes become a long list of ip4 and ip6 terms that cost nothing, and the error disappears. It works on the day you do it. The problems start afterwards:

• The provider's addresses change without telling you. An include is a promise that the provider keeps its own list current. A copied list is a snapshot. When the provider adds a new range, mail sent from it fails SPF for your domain, and nobody gets an error message: it shows up only as rising SPF failures in your DMARC reports, or as mail that quietly lands in spam.
• Not every include can be flattened. Records that use exists or macros are evaluated per message, so there is no fixed address list to copy.
• Records get long. A single TXT string holds 255 characters, so a flattened record is split into several strings. RFC 7208 also asks that the whole answer fit in a 512-octet DNS response and suggests staying under 450 octets for the name and all TXT records together. Past that, some verifiers may never see the record at all, which is a worse failure than the one you were fixing.

If you still need to flatten, flatten only what you control, such as your own servers, and keep the third-party includes. If a provider's list has to be copied, automate it: regenerate the record on a schedule from the provider's live include, compare it with what is published, and get alerted when it changes. A flattened record that nobody updates is a delayed outage.

After the fix

Run the domain through the SPF lookup counter again once the new record is live. Resolvers can keep the old record until its TTL runs out, so if the count has not changed yet, wait and check again rather than editing further. Make sure there is still exactly one SPF record at the root, because two SPF records fail just as hard as too many lookups.

Then watch your DMARC reports for a week or two. Every service you kept should show SPF passing for its return-path, and services you moved to a subdomain should show the subdomain there. If one of them now fails SPF but still passes DKIM aligned, DMARC passes and nothing is lost, but it means a step above removed something the service needed.

With Patchkit, a monitored domain whose record goes over the limit raises an alert that names the number of lookups it needs, and the alert resolves on its own once a later check finds the record back under 10. Any change to the SPF record raises a separate alert, so a colleague's new include does not go unnoticed. The SPF record generator writes a clean record and counts the includes it adds, and if you are working toward DMARC enforcement, the staged plan from p=none to p=reject shows where a healthy SPF record fits.