[Yuri989]

legit_bots/meta.json verifies Meta's fetchers with the forward-confirmed fbsv.net rDNS (AS32934 — e.g. 69.63.184.10 → fwdproxy-ncg-010.fbsv.net) and matches four user-agent tokens. Meta's Meta Web Crawlers page documents two more crawlers that belong to the same fetch fleet, and they are not matched today:

• meta-externalfetcher — "fetches individual links at a user's request", and may bypass robots.txt rules (the user-triggered preview / agentic fetch)
• meta-externalads — advertising and business-related crawl
• Meta-ExternalHit — observed in access logs; not listed on Meta's page at the time of writing, so the weakest of the three

Only the user_agent alternation changes; rdns is byte-identical to master, so each added token must still resolve to a forward-confirmed fbsv.net PTR to be exempted — the network half keeps doing the verification. Source for the documented tokens: https://developers.facebook.com/docs/sharing/webmasters/web-crawlers (checked 2026-11-05).

Why it matters: a site behind the AppSec bot challenge answers these fetchers with the challenge page, so a URL shared on Facebook/Instagram/Messenger renders without title, description or image — a failure that is silent on Meta's side.

Measured on crowdsec v1.8.2 with the bot-challenge collection loaded and the extended file in place: meta-externalfetcher + a real fetcher address (66.220.149.32 → fwdproxy-prn-032.fbsv.net) is exempted, while the same UA from an arbitrary address stays challenged. With the token absent from the current alternation, MatchKnownBot() cannot match the UA half and the fetch is challenged.

[Yuri989]

Closing as a duplicate of #503, which already adds the two tokens Meta documents (meta-externalads, meta-externalfetcher) to the same file and keeps the diff to the user_agent alternation. This PR additionally carried Meta-ExternalHit, which is not on Meta's crawler page, so #503 is the better-scoped change.

One measurement from a local crowdsec v1.8.2 AppSec (challenge collection loaded, file mounted into legit_bots/) that may be useful for #503: meta-externalfetcher/1.1 from 66.220.149.32 (fwdproxy-prn-032.fbsv.net) is exempted with the extended file, while the same UA from an arbitrary address stays challenged — the fbsv.net half keeps carrying the verification.