robots-txt-parser CDN files

jsDelivr monthly hits badge
  1. robots-txt-parser@2.0.3 /
  2. test /
  3. robots
...
can-crawl.js 2.26 KB
fetch.js 524 B
get-crawl-delay.js 1021 B
get-crawlable-links.js 843 B
get-host.js 725 B
get-sitemaps.js 883 B
is-cached.js 1.52 KB
use-robots-for.js 332 B

Free Open Source CDN for robots-txt-parser

Looking for a nice landing page for your package? https://www.jsdelivr.com/package/npm/robots-txt-parser