[Yuri989]

Meta documents five crawler agents on https://developers.facebook.com/docs/sharing/webmasters/crawler — facebookexternalhit (which also shows up as facebookcatalog), meta-webindexer, meta-externalads, meta-externalagent and meta-externalfetcher. meta.json carries four of them; this adds the two that are still missing:

• meta-externalads/1.1 (+/documentation/sharing/webmasters/web-crawlers) — "crawls the web for use cases such as improving advertising and other business-related products and services"
• meta-externalfetcher/1.1 (+/documentation/sharing/webmasters/web-crawlers) — "fetches individual links at a user's request and supports product functions such as evaluating and improving agentic AI capabilities … this crawler may bypass robots.txt rules"

No new trust is introduced: both come from the fetch infrastructure the entry already verifies. It is the same forward-confirmed rDNS fbsv.net mask, observed on four POPs — 69.63.184.10 → fwdproxy-ncg-010.fbsv.net, 173.252.127.4 → fwdproxy-frc-004.fbsv.net, 66.220.149.32 → fwdproxy-prn-032.fbsv.net, 69.171.230.114 → fwdproxy-cco-114.fbsv.net — so the entry keeps requiring the user agent and the rDNS, exactly like the four tokens already there.

Verified locally against a v1.8.2 engine with the modified file loaded: cscli bot match --ip 69.63.184.10 --user-agent "meta-externalfetcher/1.1" matches from legit_bots/meta.json; the same IP with a browser user agent does not match; a random IP with facebookexternalhit/1.1 does not match. The agent comparison is case-insensitive in practice (a capitalised Meta-ExternalAgent/1.1 matches the existing lowercase pattern), so no (?i) is needed.

Out of scope here and reported separately: the entry has no ranges, and Meta's IPv6 fetch sources have no PTR at all (dig -x returns nothing), so those fetches cannot be matched by an rdns-only entry and stay challenged — see crowdsecurity/hub#1983.

Prepared with AI assistance; every command and output above was run and checked before posting.

[Yuri989]

IPv6 side of the same fetchers — would a ranges branch be accepted here?

Meta fetches over IPv6 as well, and Meta's IPv6 sources have no PTR at all, so the rdns mask in this file cannot match them however complete the UA list is.

Measured on crowdsec v1.8.2, appsec with the challenge collection loaded, addresses claimed per request:

source	reverse PTR	verdict with this file
69.63.184.10, 66.220.149.32 (IPv4 fetch fleet)	fwdproxy-ncg-010.fbsv.net, fwdproxy-prn-032.fbsv.net	allowed
2a03:2880:11ff:5a:: (observed in production, AS32934)	none	challenged
2a03:2880:21ff:9::, 2a03:2880:15ff:4e::7, 2a03:2880:99ff::1	none	challenged

The shape I'd propose is already in this repo: telegram.json matches a vendor's whole blocks including IPv6 (2001:67c:4e8::/48, 2a0a:f280::/32, …), and commoncrawl.json, naver.json and seznam.json already carry rdns and ranges in the same entry. So: keep the rdns branch for IPv4 untouched, add ranges for Meta's announced IPv6 blocks —

{"name":"meta","user_agent":"facebookexternalhit|meta-externalagent|facebookcatalog|meta-webindexer|meta-externalads|meta-externalfetcher","rdns":["(^|\\.)fbsv\\.net$"],"ranges":["2a03:2880::/29","2a03:83e0::/32","2401:db00::/32","2620:0:1c00::/40","2803:6080::/29"]}

Coverage today, from the registry: AS32934 announces 236 IPv4 + 326 IPv6 prefixes, and 326/326 IPv6 fall inside those five blocks, 0 outside (RIPEstat announced-prefixes). The blocks come from the -i origin AS32934 route6 objects — the method Meta's own retired crawler page documented for its ranges — and the parent 2a03:2880::/32 has been announced continuously since 2013-01-03 (routing-history).

Strength is unchanged for IPv4 (the rdns half stays, so an IPv4 source still has to forward-confirm). For IPv6 the entry would trust Meta's announced space plus the UA token — the only lever, since nothing publishes PTRs there. The maintenance this implies is one registry check per period, not a watched list.

Happy to push it as a follow-up commit on this branch if that's the shape you want. If you'd rather keep the entry rDNS-only, that's fine too — I'd just like it stated, so operators know to hold the IPv6 half in a custom config.