robots-txt-parser CDN files
- robots-txt-parser@2.0.3 /
- test /
- test-data
| ... | |
| can-crawl-test-data.js | 1.56 KB |
| example-parsed-robots-short.js | 974 B |
| example-robots-malformed-short.js | 468 B |
| example-robots-txt-allow.js | 132 B |
| example-robots-txt-bbc.js | 1.85 KB |
| example-robots-txt-bcc.js | 822 B |
| example-robots-txt-blank-lines.js | 405 B |
| example-robots-txt-disallow.js | 669 B |
| example-robots-txt-karwei.js | 798 B |
| example-robots-txt-short.js | 688 B |
| example-robots-txt-sitemaps.js | 702 B |
| example-robots-txt-zalando.js | 506 B |
Free Open Source CDN for robots-txt-parser
Looking for a nice landing page for your package? https://www.jsdelivr.com/package/npm/robots-txt-parser