Learn how to configure a Grepr OpenTelemetry integration to connect with Amazon CloudWatch, enabling you to ingest logs to a Grepr pipeline for processing and send the processed logs to a CloudWatch log group. # Configure Amazon CloudWatch connections with a Grepr integration To use Amazon CloudWatch as a destination for your Grepr pipelines, use the Grepr OpenTelemetry integration to ingest logs into your Grepr pipelines for processing and send the processed logs to CloudWatch. The OpenTelemetry integration supports receiving logs from any collector that supports OTLP/HTTP, such as the OpenTelemetry Collector and Fluent Bit, and sending the processed logs to a CloudWatch log group using the CloudWatch OTLP/HTTP endpoint. You can use the Grepr UI or REST API to create an OpenTelemetry integration configured for CloudWatch. To learn how to create an OpenTelemetry integration in the Grepr UI and configure collectors to send logs to Grepr, see [Use an OpenTelemetry integration with CloudWatch in a Grepr pipeline](#use-cloudwatch-integration). > Although this page describes using an integration to create a pipeline for sourcing and sending data with a single observability platform, your pipelines can use multiple integrations to source and sink data for different platforms. For example, you can create a pipeline with a Datadog integration that's used to source data from Datadog agents, and a Splunk integration and sink that sends the pipeline output to Splunk. ## Requirements Connecting Grepr to CloudWatch uses CloudWatch's OTLP/HTTP endpoint, and requires the following: * The CloudWatch OTLP logs endpoint for your region. * An Amazon CloudWatch log group and log stream. The log group must have bearer token authentication enabled. * An AWS IAM user with `logs:PutLogEvents` permission on the CloudWatch log group and a bearer token to authenticate the user to CloudWatch. To learn more about the CloudWatch support for OTLP/HTTP, see [OTLP Endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html). To learn more about creating an IAM user, bearer token, and the log group and stream for CloudWatch's OTLP/HTTP endpoint, see [Setting up bearer token authentication](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_HTTP_Endpoints_BearerTokenAuth.html). ## Use an OpenTelemetry integration with CloudWatch in a Grepr pipeline \[#use-cloudwatch-integration] ### Step 1: Create an OpenTelemetry integration To create an OpenTelemetry integration configured for CloudWatch in the Grepr UI: 1. Go to the **Integrations** page in the Grepr UI and click **Add new** next to **Observability Vendors**. 2. In the **Add Observability Vendor** dialog: * In the **Type** menu, select **Open Telemetry**. * In the **Name** field, enter a name for the integration. Grepr recommends naming it descriptively so it's easy to identify. For example, `cloudwatch-prod-logs`. * In the **Base Endpoint URL** field, enter `https://logs..amazonaws.com`, replacing `` with your AWS region. * Click **Add Header** and add the following three headers. For the `Authorization` header, select the lock icon to securely store the token. * **Header Name**: `Authorization`. **Header Value**: `Bearer `. Replace `` with the bearer token for the CloudWatch IAM user. * **Header Name**: `x-aws-log-group`. **Header Value**: the name of your CloudWatch log group. * **Header Name**: `x-aws-log-stream`. **Header Value**: the name of your CloudWatch log stream. To send logs from a single integration to dynamically chosen log groups or log streams based on log content, configure `x-aws-log-group` or `x-aws-log-stream` as attribute-sourced headers instead of static values. * In the **Output encoding** menu, select **JSON**. 3. Click **Create**. 4. Save the `GREPR TOKEN` displayed in the confirmation dialog securely. #### Optional: Grant team access to the integration When you have teams in your Grepr organization, you can grant access to the integration to one or more of those teams. To configure team access when you create or edit an integration: 1. In the **Grant Team Access** section, click the plus sign icon (+). 2. Select the team or teams that should be granted access to the integration. 3. Click **Create**. To learn more, see [Manage teams in the Grepr platform](/admin/manage-teams) and [Permissions in the Grepr platform](/grepr-platform/permissions). CloudWatch's OTLP endpoint determines the destination log group and log stream from the `x-aws-log-group` and `x-aws-log-stream` request headers. If you configure static values for these headers, your Grepr integration sends every log event to the same log group and log stream. To route log events from your integration to dynamically chosen CloudWatch log groups or log streams based on attributes included in the events, select **From Attribute** and enter the **Attribute Path** value when you configure the `x-aws-log-group` or `x-aws-log-stream` headers. Grepr groups log events that share the same attribute value into the same outbound request, sets the header to that value, and forwards the events to CloudWatch. For example, to send log events from a Kubernetes namespace to a log stream named after the namespace: 1. In the Grepr OpenTelemetry integration, edit the `x-aws-log-stream` header. 2. In the **Source** menu, select **From Attribute**. 3. In the **Attribute Path** field, enter the path to the attribute that holds the namespace. For example, `kubernetes.namespace.name`. 4. Save the header. When a log event is missing the configured attribute, Grepr still delivers the event to the CloudWatch endpoint, but without the `x-aws-log-stream` header. Because the CloudWatch OTLP endpoint relies on this header to determine the destination log stream, ensure every log event routed through the integration contains the required attribute, or use a static value for `x-aws-log-stream`. ### Step 2: Add a source to your pipeline To add a source to your pipeline that uses the OpenTelemetry integration: 1. In the Grepr UI, go to the details page of an existing pipeline or create a new pipeline from the homepage. 2. In the left-hand navigation menu, click **Sources**. 3. In the **Add source** dialog, select your OpenTelemetry integration in the **Source** menu. 4. Click **Submit**. ### Step 3: Configure log forwarding to Grepr To send data to Grepr, you configure your collector or agent to send it to the ingestion URL for your Grepr integration, rather than to your observability vendor's platform. To find the ingestion URL when using the Grepr UI to configure your pipeline, go to the details page for the pipeline, click **Sources** in the left pane, and copy the **Ingest URL**. If you connect to Grepr through AWS PrivateLink, copy the private ingestion URL and use it in place of the public ingestion URL. To learn more about sending data to Grepr over AWS PrivateLink, see [Securely forward observability data to Grepr with AWS PrivateLink](/admin/private-link). When you use the API, you can construct the ingestion URL using the following format: `https://-.ingest.grepr.ai/otlp` Replace `` with the identifier of the vendor integration used by your pipeline and replace `` with the identifier for your organization. To use the PrivateLink endpoint service, insert `.private` after the `ingest` host segment. Although all sources created from the same integration share the same ingestion URL, you can use a single integration to create sources for multiple pipelines. The Grepr platform ensures events are routed to the correct pipelines. Authenticate by sending the Grepr token in the `x-grepr-token` header. Replace `` in the examples below with the token you saved after creating the OpenTelemetry integration in Step 1. To see more information and configuration examples, select the tab for your preferred log collector: The OpenTelemetry Collector can send logs to Grepr using the `otlphttp` exporter. To configure the OpenTelemetry Collector to send logs to Grepr's ingestion endpoint, update your collector configuration (`collector.yaml`): ```yaml exporters: otlphttp: endpoint: headers: x-grepr-token: service: pipelines: logs: receivers: [otlp] processors: [batch] exporters: [otlphttp] ``` Replace `` with the full ingestion URL provided by the Grepr UI, and `` with your Grepr token. To learn more about OpenTelemetry Collector configuration, see the [OpenTelemetry Collector documentation](https://opentelemetry.io/docs/collector/). Fluent Bit can forward logs to Grepr using the OpenTelemetry output plugin. To configure the plugin: * Set the `host` value using only the hostname of the ingestion URL from your OpenTelemetry integration, omitting the `https://` prefix and the path value. * Set the `logs_uri` value to the ingestion URL path and append `/v1/logs`. * Add a `header` entry that sends the Grepr token as `x-grepr-token`. The following is an example `OUTPUT` configuration for the OpenTelemetry plugin from a Fluent Bit configuration file (`fluent-bit.conf`): ```ini [OUTPUT] name opentelemetry match * host -.ingest.grepr.ai port 443 logs_uri /otlp/v1/logs header x-grepr-token tls On tls.verify On ``` To learn more about configuration settings for Fluent Bit, see the documentation for the [Fluent Bit OpenTelemetry output plugin](https://docs.fluentbit.io/manual/data-pipeline/outputs/opentelemetry). ## Frequently Asked Questions ### How do I connect Grepr to Amazon CloudWatch? Create an OpenTelemetry integration in Grepr with the CloudWatch OTLP endpoint, https\://logs.\.amazonaws.com, and configure three headers: an Authorization header with a bearer token stored securely with the lock icon, an x-aws-log-group header with the log group name, and an x-aws-log-stream header with the log stream name. Set Output encoding to JSON. You must enable bearer token authentication on the log group in AWS before sending logs. ### Why does Grepr require bearer token authentication for CloudWatch? CloudWatch's OTLP endpoint requires bearer token authentication rather than standard IAM access keys. The bearer token authenticates an IAM user that has logs:PutLogEvents permission on the target log group. ### What output encoding should I use for the CloudWatch integration? Use JSON. When you create the OpenTelemetry integration for CloudWatch, select JSON in the Output encoding menu so Grepr sends log events to the CloudWatch OTLP endpoint in JSON format. ### Does Grepr create CloudWatch log groups automatically? No. You must create the CloudWatch log group and log stream in AWS before Grepr can send logs to them. You must also enable bearer token authentication on the log group. ### Can I use an existing CloudWatch log group with Grepr? Yes. Enable bearer token authentication on the existing log group, and create the log stream if it does not already exist. ### How do I route logs to different CloudWatch log groups or log streams? Configure the x-aws-log-group or x-aws-log-stream header as an attribute-sourced header. In the header configuration, select From Attribute and enter the Attribute Path for the attribute that holds the destination value, such as kubernetes.namespace.name. Grepr groups events that share the same attribute value into the same request and sets the header to that value. If an event is missing the attribute, Grepr delivers it without that header, so use a static value when an event might not include the attribute.