# HyperDX OpenTelemetry Node

OpenTelemetry Node Library for [HyperDX](https://www.hyperdx.io/)

### Install HyperDX OpenTelemetry Instrumentation Package

Use the following command to install the OpenTelemetry package.

```sh
npm install @hyperdx/node-opentelemetry
```

or

```sh
yarn add @hyperdx/node-opentelemetry
```

### Add The Logger Transport

To collect logs from your application, you'll need to add a few lines of code to
configure your logging module.

#### Winston Transport

```ts
import winston from 'winston';
import * as HyperDX from '@hyperdx/node-opentelemetry';

const MAX_LEVEL = 'info';

const logger = winston.createLogger({
  level: MAX_LEVEL,
  format: winston.format.json(),
  transports: [
    new winston.transports.Console(),
    HyperDX.getWinstonTransport(MAX_LEVEL), // append this to the existing transports
  ],
});

export default logger;
```

#### Pino Transport

```ts
import pino from 'pino';
import * as HyperDX from '@hyperdx/node-opentelemetry';

const MAX_LEVEL = 'info';

const logger = pino({
  mixin: HyperDX.getPinoMixinFunction,
  transport: {
    targets: [
      HyperDX.getPinoTransport(MAX_LEVEL),
      // other transports
    ],
  },
});

export default logger;
```

### Configure Environment Variables

Afterwards you'll need to configure the following environment variables in your
shell to ship telemetry to HyperDX:

```sh
export HYPERDX_API_KEY=<YOUR_HYPERDX_API_KEY_HERE> \
OTEL_SERVICE_NAME='<NAME_OF_YOUR_APP_OR_SERVICE>'
```

_Self-hosted users will need to additionally specify the `OTEL_EXPORTER_OTLP_ENDPOINT` env var to point to their self-hosted endpoint (ex. `http://localhost:4318`)_

### Run the Application with HyperDX OpenTelemetry CLI

#### Option 1 (Recommended)

Now you can run the application with the HyperdxDX `opentelemetry-instrument`
CLI.

```sh
npx opentelemetry-instrument index.js
```

#### Option 2

In case you want to run the application with a custom entry point (nodemon,
ts-node, etc.).

Run your application with the following command (example using `ts-node`):

```sh
npx ts-node -r '@hyperdx/node-opentelemetry/build/src/tracing' index.ts
```

#### Option 3

You can also manually instrument the SDK. In the `instrument.ts`, add the following code:

```ts
import { initSDK } from '@hyperdx/node-opentelemetry';

initSDK({
  consoleCapture: true, // optional, default: true
  additionalInstrumentations: [], // optional, default: []
  additionalResourceAttributes: { // optional, default: {}
    // Add custom resource attributes to all telemetry data
    'environment': 'production',
    'deployment.version': '1.0.0',
    'custom.attribute': 'value'
  },
});

// Other instrumentation code...
// Details link: https://opentelemetry.io/docs/instrumentation/js/manual/#manual-instrumentation-setup
```

And run your application with the following command (example using `ts-node`):

```sh
npx ts-node -r './instrument.ts' index.ts
```

### Troubleshooting

If you are having trouble getting events to show up in HyperDX, you can enable verbose logging by setting the environment variable `OTEL_LOG_LEVEL=debug`. This will print out additional debug logging to isolate any issues.

If you're pointing to a self-hosted collector, ensure the `OTEL_EXPORTER_OTLP_ENDPOINT` environment variable is set to the correct endpoint (ex. `http://localhost:4318`) and is reachable (ex. `curl http://localhost:4318/v1/traces` should return a HTTP 405).

#### Bundlers (esbuild, webpack, etc.)

When using bundlers like esbuild with OpenTelemetry, in order to auto-instrument supported libraries, you'll want to make sure you make them available outside of the bundle. For instance:

- For **esbuild**: Use a plugin like [esbuild-node-externals](https://www.npmjs.com/package/esbuild-node-externals) to exclude Node.js modules and libraries from bundling
- For **other bundlers**: Use equivalent plugins or configurations that prevent bundling of Node.js modules and dependencies that OpenTelemetry needs to instrument

This ensures that OpenTelemetry can properly access and patch the original modules at runtime rather than using bundled versions, which would make them unavailable to auto-instrumentation.

### (Optional) Attach User Information or Metadata (BETA)

> WARNING: ONLY WORKS WITH NODE 14.8.0+

To easily tag all events related to a given attribute or identifier (ex. user id or email), you can call the `setTraceAttributes` function which will tag every log/span associated with the current trace after the call with the declared attributes. It's recommended to call this function as early as possible within a given request/trace (ex. as early in an Express middleware stack as possible).

This is a convenient way to ensure all logs/spans are automatically tagged with the right identifiers to be searched on later, instead of needing to manually tagging and propagating identifiers yourself.

`userId`, `userEmail`, `userName`, and `teamName` will populate the sessions UI with the corresponding values, but can be omitted. Any other additional values can be specified and used to search for events.

```sh
export HDX_NODE_BETA_MODE=1
```

```ts
import { setTraceAttributes } from '@hyperdx/node-opentelemetry';

app.use((req, res, next) => {
  // Get user information from the request...

  // Attach user information to the current trace
  setTraceAttributes({
    userId,
    userEmail,
  });
  next();
});
```

### (Optional) Advanced Instrumentation Configuration

#### Adding Custom Resource Attributes

Resource attributes are key-value pairs that describe the resource (service/application) producing telemetry data. These attributes are attached to all traces, metrics, and logs exported from your application. Common use cases include adding environment information, deployment versions, or custom metadata for filtering and grouping telemetry data.

When using manual instrumentation with `initSDK`, you can add custom resource attributes using the `additionalResourceAttributes` parameter:

```ts
import { initSDK } from '@hyperdx/node-opentelemetry';

initSDK({
  additionalResourceAttributes: {
    'deployment.environment': process.env.NODE_ENV || 'development',
    'service.version': process.env.APP_VERSION || '0.0.0',
    'service.namespace': 'my-namespace',
    'cloud.region': process.env.AWS_REGION,
    // Add any custom attributes your organization needs
    'team.name': 'backend-team',
    'feature.flag': 'new-checkout-flow'
  },
});
```

These attributes will be included with all telemetry data sent to HyperDX, making it easier to filter and analyze your observability data. For standard attribute names, refer to the [OpenTelemetry Resource Semantic Conventions](https://opentelemetry.io/docs/specs/semconv/resource/).

#### Adding Additional 3rd-Party Instrumentation Packages

When manually instrumenting the SDK, use the `additionalInstrumentations` key to create an array of additional 3rd-party instrumentations. Check [here](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/metapackages/auto-instrumentations-node#supported-instrumentations) to see the current automatically instrumented packages.

```js
// In your instrument.js/ts file...
const { initSDK } = require('@hyperdx/node-opentelemetry');
const { RemixInstrumentation } = require('opentelemetry-instrumentation-remix');

initSDK({
  consoleCapture: true, // optional, default: true
  additionalInstrumentations: [new RemixInstrumentation()], // your custom instrumentations here
});
```

You can then use the instrumentation file above by requiring it when starting your application:

```sh
node -r './instrument.js' index.js
```

or see [manual instrumentation steps](#option-3) for more options.

#### Capture Console Logs

By default, the HyperDX SDK will capture console logs.
You can disable it by setting `HDX_NODE_CONSOLE_CAPTURE` environment variable to 0.

```sh
export HDX_NODE_CONSOLE_CAPTURE=0
```

To attach trace id to console logs, you can set `HDX_NODE_BETA_MODE` environment variable to 1.

```sh
export HDX_NODE_BETA_MODE=1
```

#### Advanced Network Capture (BETA)

By enabling advanced network capture, the SDK will additionally capture full HTTP request/response headers
and bodies for all inbound/outbound HTTP requests, to help with more in-depth request debugging.
This can be accomplished by setting `HDX_NODE_ADVANCED_NETWORK_CAPTURE` environment variable to 1.

```sh
export HDX_NODE_ADVANCED_NETWORK_CAPTURE=1
```

By default, all request/response headers will be captured. You can specify a custom list of headers to capture
by setting `OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_CLIENT_REQUEST`,
`OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_CLIENT_RESPONSE`,
`OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_REQUEST`,
`OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_SERVER_RESPONSE`
environment variable to a comma-separated list of headers.

For example:

```sh
export OTEL_INSTRUMENTATION_HTTP_CAPTURE_HEADERS_CLIENT_REQUEST=authorization,accept
```

### Custom Termination/Shutdown Handler

If your application specifies its own shutdown procedure by capturing `SIGTERM` or `SIGINT` events, you'll want to disable the default shutdown handler by specifying `stopOnTerminationSignals` to false in the `initSDK` call or by setting `HDX_NODE_STOP_ON_TERMINATION_SIGNALS` to `false`.

You can then import and call the `shutdown` async function to manually stop the SDK.

```ts
import { shutdown } from '@hyperdx/node-opentelemetry';

process.on('SIGTERM', async () => {
  // Your shutdown code here...

  await shutdown();
  process.exit();
});
```

### GCP Cloud Function Event Handler

If your application runs on GCP (Google Cloud Platform) Cloud Functions, you can use the `registerGCPCloudFunctionEventHandler` function to automatically instrument your function.
Currently the event handler will ensure traces gets propagated through PubSub as long as the publisher has `enableOpenTelemetryTracing: true` (see [example](https://github.com/googleapis/nodejs-pubsub/blob/730e9dc0ab20a5f02508a82310d58b0da1f54330/samples/openTelemetryTracing.js#L86)).

```ts
import functions from '@google-cloud/functions-framework';
import { registerGCPCloudFunctionEventHandler } from '@hyperdx/node-opentelemetry';

functions.cloudEvent(
  'helloCloudEvent',
  registerGCPCloudFunctionEventHandler(async (event) => {
    // Your code here...
  }),
);
```