# Redis Recovery

This sub-repo contains tools for recovering the state of the destination sender redis servers from data
stored in the kafka clearing topics

## Requirements

This application requires node.js version 11 or 12, plus the Yarn package manager.

## Installation

```sh
# clone the repo
git clone git@github.com:bitbrew/hub-tools
# install packages
cd hub-tools/redis-recovery
yarn
```

Or install with npm or yarn 

```sh
# Using npm
npm i -g bbredis-recovery
# Using yarn
yarn global add bbredis-recovery
```

## Usage

```
# Working within the repository
./index.js

# Working with the globally installed tool
redis-recovery
```

All examples will use 'redis-recovery' rather than the local js file.

### Getting help

```sh
redis-recovery --help
```

### Configure Kafka Settings

```sh
redis-recovery kafka-config 127.0.0.1 9092 test-topic 50
```

### Configure Redis Settings

```sh
redis-recovery redis-config 127.0.0.1 6379 Keen
```

### Display Current Config

```sh
redis-recovery config
```

### Export Recovery Data From Kafka

```sh
redis-recovery kafka-export
```

This may take a number of seconds or minutes to complete.  When completed the output to the console will look similar to:

```sh
Using the following config:
Kafka topic:       test-topic
Kafka partitons:   50
Kafka host:        127.0.0.1
Kafka port:        9092
Kafka log level:   2
Redis host:        127.0.0.1
Redis port:        6379
Redis destination: Keen

✔ Initializing Kafka consumer
✔ Getting retry batches not followed by closed batches
✔ Processed 100106 records
✔ Retrieving count of queued detinations
✔ Processed 100106 records
✔ Writing results to output-kafka.json
```

### Export Redis State

```sh
redis-recovery redis-export
```

This should take only a few seconds, following which the console output will be similar to:

```
Using the following config:
Kafka topic:       test-topic
Kafka partitons:   50
Kafka host:        127.0.0.1
Kafka port:        9092
Kafka log level:   2
Redis host:        127.0.0.1
Redis port:        6379
Redis destination: Keen

✔ Retrieving retry batches
✔ Writing results to output-redis-retry.json
✔ Retriving clearing state
✔ Writing results to output-redis-clearing.json
```

### Import Redis State

```sh
redis-recovery redis-import
```

This will only take a few seconds to complete.  The output will give a list of the destination-
partition's that were updated in Redis:

```sh
Using the following config:
Kafka topic:       test-topic
Kafka partitons:   50
Kafka host:        127.0.0.1
Kafka port:        9092
Kafka log level:   2
Redis host:        127.0.0.1
Redis port:        6379
Redis destination: Keen

✔ Loading update file output-kafka.json
✔ Updating destination-partition 3d9eb6eb-3698-4f00-bfc7-a03d49b1efb7:49
✔ Updating destination-partition f07f8701-3ca5-4d4d-b896-3bccfea68d25:49
```