# Crawler for web accessibility

This is a web accessibility crawler written in Node.js

## Getting Started

To get started with the crawler, follow these steps:

1. Install dependencies with `npm install`
2. Run the crawler for a domain with `DOMAIN=<domain> npm run crawl`. For example, `DOMAIN=deranged.dk npm run crawl` runs for `deranged.dk` and `www.deranged.dk`. `DOMAIN` defaults to `digst.dk`.
  - You can also set a env var `LIMIT` to limit the number of pages crawled in the crawl. E.g. `DOMAIN=kbhbilleder.dk LIMIT=10000 npm run crawl` to limit to ten thousand pages that finish evaluation (skipped or failed pages do not count towards the total).

## Running on Ubuntu

When running the crawler locally on ubuntu, you can get an error because of AppArmor which restricts puppeteer.

You need to run the following script, where `CHROMIUM_BUILD_PATH` is the path to puppeteer's chrome. Example: `/home/user/.cache/puppeteer/chrome/linux-140.0.7339.80/chrome-linux64/chrome`. The correct path can be found in the AppArmor error message.
```
export CHROMIUM_BUILD_PATH=/@{HOME}/chromium/src/out/**/chrome
cat | sudo tee /etc/apparmor.d/chrome-dev-builds <<EOF
abi <abi/4.0>,
include <tunables/global>

profile chrome $CHROMIUM_BUILD_PATH flags=(unconfined) {
  userns,

  # Site-specific additions and overrides. See local/README for details.
  include if exists <local/chrome>
}
EOF
sudo service apparmor reload  # reload AppArmor profiles to include the new one
```
