# Migrating from Haraka v1.x to v2.x

Haraka v2.x contains two significant changes to the v1.x API related to streams.

Streams are an abstraction over a data flow that is provided by Node core and is used throughout node to "pipe" data between two places or more. This makes programming very easy, and is hence why we started using them in Haraka starting with version 2.0.0.

For more information about the Stream API, see http://nodejs.org/api/stream.html

Note that when using bundled Haraka plugins, it's very unlikely you will need to change anything. Though you may want to configure `spool_dir` and `spool_after` in `config/smtp.ini` (v2.x), or in `config/connection.ini` (v3.1 or newer). If you have custom plugins, continue reading.

## Changes To Look For

Firstly, the incoming data in an email (the email body) is now stored in an object which you can treat as a ReadableStream. To find if this is relevant for you, look for instances of `data_lines` in your plugins.

Secondly, if you parse the mail body, attachments are now provided as a stream, rather than custom start/data/end events. To find if this is relevant for you, look for instances of `attachment_hooks` in your plugins.

## Fixing data_lines plugins

Any plugins now working on each line of data will need to change to using a stream. The stream is called `transaction.message_stream`.

These changes may be complicated if you are iterating over each line and doing something with the strings therein. However if you are piping the data to an application or over a network, your code will become significantly simpler (and a lot faster).

In v1.x Haraka populated the `transaction.data_lines` array for each line of data received. If you were writing the data to a socket then you had to handle backpressure manually by checking the return of `write()` and adding `on('drain')` handlers like so:

    var data_marker = 0;
    var in_data = false;
    var end_pending = true;
    var send_data = function () {
        in_data = true;
        var wrote_all = true;
        while (wrote_all && (data_marker < connection.transaction.data_lines.length)) {
            var line = connection.transaction.data_lines[data_marker];
            data_marker++;
            wrote_all = socket.write(Buffer.from(line.replace(/^\./, '..').replace(/\r?\n/g, '\r\n')), 'binary');
            if (!wrote_all) return;
        }
        // we get here if wrote_all still true, and we got to end of data_lines
        if (end_pending) {
            end_pending = false;
            // Finished...
            socket.send_command('dot');
        }
    };
    socket.on('drain', function () {
        if (end_pending && in_data) {
            setImmediate(function () { send_data() });
        }
    });

In v2.x this now becomes:

    connection.transaction.message_stream.pipe(socket, {ending_dot: true});

This automatically chunks the data, handles backpressure and will apply any
necessary format changes. See `docs/Transaction.md` for the full details.

If you need to handle the input data by line, then you will need to create
your own writable stream and then pipe the message to the stream and then
extract the lines from the stream of data. See the `dkim` plugin for
an example.

## Fixing attachment_hooks plugins

For v1.x you passed in functions to `transaction.attachment_hooks()` as
follows:

    transaction.attachment_hooks(
        function (ctype, filename, body) {...}, // start
        function (buf) {...}, // data
        function () {...} // end
    );

That has now changed to:

    transaction.attachment_hooks(
        function (ctype, filename, body, stream) {...}, // start
    );

This allows you to attach the stream to other streams via `stream.pipe(dest)`.

Sometimes destination streams will apply backpressure on the sending stream,
for example if you are sending attachments to a remote service. In order
for this backpressure to apply to the connection itself (so that we don't
have to buffer up data in memory), we need to provide the connection object
to the stream:

    var transaction = connection.transaction;
    transaction.attachment_hooks(
        function (ctype, filename, body, stream) {
            stream.connection = connection;
            ...
        }
    );

For a full example of using attachment streams, see the Transaction.md
documentation file.