# deprecation-crawler

## Preconditions

1. Setup the target project.

Do this by checking out the target project e.g. RxJS

```bash
git clone git@github.com:ReactiveX/rxjs.git rxjs`
```

2. Navigate to the root of the repository.

```bash
cd rxjs
```

## Usage


1. In the root of the repository run the following command.

```bash
npx deprecation-crawler
```

Possible CLI params are:
- path (--path <path>, -p <path>)
  Path to the `deprecation-crawler.config.json`. 
  Pro Tip: use `./GIT_IGNORED_FOLDER/deprecation-crawler.config.json` for a 'dry-run'.
- tag (--tag <path>, -t <path>)
  Git tag to crawl. 
  Pro Tip: use the current branch name for a 'dry-run' (git branch --show-current).


CLI Questions  
The command will ask you a couple of questions regarding the included file locations and defaults 
to run the crawling and grouping process. It also includes helpful information.


### Setup

#### Setup `tsconfig` (one-time)

- _√ What's the location of the ts config file?_
  By default `./tsconfig.json` is set as tscinfig to extend from. These locations should represent the projects ts config settings used to determine the folders to crawl.
  Pro Tip: The result is stored in `deprecation-crawler.tsconfig.json` go there to edit included files.
  
#### Configuring the crawling process (one-time)

- _√ What's the output directory?_
  By default `./depercations` is set. This will put the resulting data into the named folder.
  Pro Tip: Move it in a gitignored folder for a 'dry-run'.
  
- _√ What's the deprecation keyword to look for?_
  By default `@deprecated` is set. Looks for comments with this keyword to add to the deprecations list.

- _√ What's the deprecation link to the docs (the deprecation ruid will be appended to this)?_
  By default `https://rxjs.dev/deprecations` is set. Used to add a link in the sourcecode to this location.
  If a deprecation comment already has this link, it will be skipped.


### Crawl

After that, the process should start crawling and every crawled file should get logged to the console.
`Looking for deprecations in path/to/file`

- _√ What git tag do you want to crawl?_
  By default `main` is set.
  Pro Tip: Pass the tag name as cli param to save repetitive questions (npx deprecation-crawler --tag main)

For all crawled deprecations ruis get generated. 

### Sync Repository

- _√ Do you want to sync the repo?_
 This commits the changes to the version control
 Pro Tip: Edit the commit message under `config#defaultCommitMessage` 

### Group

After the crawling is done, a new message should show up in the console saying
`Adding grouping to deprecations...`

You will get asked 2 questions for every crawled deprecation:

- _√ Add group to deprecation path/to/file#LINE_NUMBER_
  The text of the deprecation message gets listed and you have to enter a string for the group name.
  By default `ungrouped` is suggested. These strings serve as a reverence to the group.

- _√ Add regexp to group_
  This question asks for a regular expression used to check every new deprecation against it is to see if it matches the group's conditions/regexes.
  Every group can have multiple regular expressions to test a deprecation for.

The deprecation message as well as the passed reges string will get normalized

- trim white spaces
- transform multiple white space to one
- all lowercase

Examples for message `The full deprecation message for {@link test} thingy!`:

- Full message `The full deprecation message for {@link test} thingy!`
- Partial message/Includes `full deprecation`

Pro Tip:
Use groups to:
- Address health checks e.g. deprecations without a message
- Maintain custom content for a group
- Validate the messages against a set of patterns

If you just hit enter no regular expression gets saved.

This means that every deprecation gets its own ruid generated by created from its function signature.
Doing this enables us to detect already crawled deprecations, malicious deprecations as well as a ruid across machines and codebases/repositories.

After the deprecations have been processed, the source code of the repository will be updated. A link to the deprecation info will be added at the end of the deprecation message.

### Format

There are currently 2 formatters built in:
- tag-based markdown
- group-based markdown

Both are set as default in the setup step.

- _√ Update Formats?_
  Hit enter to generate the registered formats.

Check if the generated configs ended up in `./[config#outputPath]/raw-deprecations.json`.
And the crawled deprecations are present in the configured folder (by default in the `./deprecations` folder).

Pro Tip: edit the formatters under `[config#formatters]`

## Usage during development

### Run deprecation-crawler against the [sandbox project]('./sandbox')

Run the command:

For npm:

```bash
npm run test
```

For yarn:

```bash
yarn test
```

This will:

- build the project
- setup the sandbox (copy input files as to-be crawled files)
- run the crawler against these files
- verify that: cli output is correct, deprecation notes are added to the repository, the output JSON file is correctly generated

### Run deprecation-crawler against a real repository

1. Build the deprecation finder:

For npm:

```bash
npm run build
```

For yarn:

```bash
yarn build
```

2. Copy path to the packed file:

   Open the `dist/packages/deprecation-crawler` folder and search for the `index.js` file.
   copy the absolute path to `index.js`.

3. Navigate to the root of your target project

```bash
cd path/to/the/root/folder
```

4. Execute the packed file by using the absolute path copied in step 2.

```bash
npx path/to/deprecation-crawler/dist/index.js
```

5. go on with the step 2 form section [Usage](#Usage)