# gamecomp.js
Copyright 2010-2021 Adam Nielsen <<malvineous@shikadi.net>>  

This is a Javascript library that can pass data through different algorithms
used by MS-DOS games from the 1990s.  Typically this is used to compress and
decompress game data, as well as encrypt and decrypt it too.

## Supported algorithms

### Compression

* cmp-bpe-stargunner: Stargunner *.DLT byte-pair encoding
* cmp-carmackize: id Software Carmackization compression
* cmp-lzexe: LZEXE .exe files (decompression only, equivalent to UNLZEXE)
* cmp-lzss: Generic LZSS (Lempel-Ziv-Storer-Szymanski)
  * Byte mode (8 flag bits stored upfront in a single byte)
  * Bit mode (each flag bit stored before each literal or length+distance code)
* cmp-lzw: Generic LZW (Lempel-Ziv-Welch)
* cmp-pklite: PKLite .exe files (decompression only)
* cmp-rle-bash: Monster Bash *.DAT run-length-encoding
* cmp-rle-ccomic: Captain Comic *.EGA run-length-encoding
* cmp-rlew-id: id Software RLEW encoding used in later games (Keen 4 and later)
* cmp-rle-id: id Software RLE encoding used in early games (Keen 3 and earlier)

### Encryption

* enc-bpa-drally-filename: Death Rally *.BPA filename encryption
* enc-dlt-stargunner-filename: Stargunner *.DLT filename encryption
* enc-glb-raptor: Raptor *.GLB cipher
* enc-xor-blood: Monolith/Blood *.RFF XOR cipher
* enc-xor-incremental: Generic XOR with initial seed, step and limit

### Other

* pad-generic: Add/remove padding bytes at repeating intervals to account for
  bugs in the way some games load and save files.
* pad-chunked: Apply an algorithm repeatedly over blocks of data in a file.
  Used for cases where RLE algorithms cannot cross a 64 kB boundary, by
  splitting the data into 64 kB chunks and applying the RLE or other algorithm
  independently on each chunk.

## Installation as an end-user

If you wish to use the command-line `gamecomp` utility to work with the
algorithms directly, you can install the CLI globally on your system:

    npm install -g @camoto/gamecomp-cli

For Arch Linux users the AUR package `gamecomp-cli` is also available.

### Command line interface

The `gamecomp` utility can be used to apply and reverse algorithms on data.
Data to process is supplied on `stdin` and the processed data is sent to
`stdout`.  Use the `--help` option to get a list of all the available options.
Some quick examples:

    # List supported algorithms and their options
    gamecomp --formats

    # Compress a file using LZW with some custom options
    gamecomp +cmp-lzw cwEOF=256 cwFirst=257 < clear.txt > out.lzw

    # Decrypt a file with an XOR cipher using the default options
    gamecomp -enc-xor-blood < crypt.bin > clear.bin

When specifying the algorithm in the first parameter, it is prefixed with a `+`
to apply the algorithm (compress/encrypt) or a `-` to reverse it
(decompress/decrypt).

## Installation as a dependency

If you wish to make use of the library in your own project, install it
in the usual way:

    npm install @camoto/gamecomp

See `cli/index.js` for example use.  The quick start is:

    import { cmp_lzw, enc_xor_blood } from '@camoto/gamecomp';
    
    // Decompress a file
    const input = fs.readFileSync('data.lzw');
    const output = cmp_lzw.reveal(content);
    fs.writeFileSync('data.raw', output);
    
    // Encrypt the file with custom options
    const output = enc_xor_blood.obscure(input, {
        seed: 123,
    });
    fs.writeFileSync('data.xor', output);

## Installation as a contributor

If you would like to help add more file formats to the library, great!
Clone the repo, and to get started:

    npm install

Run the tests to make sure everything worked:

    npm test

You're ready to go!  To add a new algorithm:

 1. Create a new file in the relevant subfolder for the algorithm type, such as
    `compress/` or `encrypt/`.

 2. In the `compress/` or `encrypt/` folder, edit `index.js` and add a line for
    your new file.

 3. Make a folder in `test/` for your new algorithm and populate it with
    files similar to the others.  The tests work by passing standard data to
    each algorithm and comparing the result to what is inside this folder.  Run
    the tests just for your new algorithm (instead of all of them) by passing
    the grep (`-g`) parameter to Mocha, the test framework.  This will run any
    test matching the given string:
    
        npm test -- -g cmp-myformat
    
    Your tests will fail until you have created the expected sample files in
    the `test/cmp-myformat/` folder.
    
    You can either create these files by hand, with another utility, or if you
    are confident that your code is correct, from the code itself.  This is done
    by setting an environment variable when running the tests, which will cause
    the data produced by your code to be saved to a temporary file in the
    current directory:
    
        SAVE_FAILED_TEST=1 npm test -- -g cmp-myformat
        cd test/cmp-myformat/ && mv default.bin.failed_test_output default.bin
    
    If you wish to run more than the standard tests, create a separate file in
    the `test/` folder named like `test-cmp-myformat.js`.  Copy the content from
    one of the existing files as an example.  As the standard tests are fairly
    basic and won't test edge cases in most algorithms, it is a good idea to
    create extra tests to cover these cases.
    
    See `test/test-enc-glb-raptor.js` for examples that load more test files
    from the same directory that the standard tests use, or
    `test/test-cmp-rle-bash.js` for simpler tests that only need to use a small
    array of data bytes.

During development you can examine the output of your algorithm like this:

    # Decompress (remove algo/reveal data)
    $ DEBUG='gamecomp:cmp-myformat*' ./bin/gamecomp.js -cmp-myformat param=value < compressed.bin > clear.test

    # Compress (apply algo/obscure data)
    $ DEBUG='gamecomp:cmp-myformat*' ./bin/gamecomp.js +cmp-myformat param=value < clear.bin > compressed.test

If you use `debug()` rather than `console.log` then these messages can be left
in for future diagnosis as they will only appear when the `DEBUG` environment
variable is set appropriately.