--- title: "Compact" description: "lossless wire-format volume reduction; expanded at query time by analyzer-side plugins or Retriever" source: "https://github.com/log-10x/modules/tree/main/apps/receiver/app.yaml" icon: "material/arrow-collapse-horizontal" --- Cut log analytics and storage costs, where the destination supports it, by losslessly compacting log/trace events at the edge before shipping them to Splunk or self-hosted Elasticsearch/OpenSearch, or to object storage paired with Retriever. Compact is one of the [Receiver](../index.md)'s per-pattern actions, alongside pass, sample, tier_down, offload, and drop. Events ship as a compact wire-form that downstream analyzer plugins or [Retriever](../../retriever/) expand at query time.

Cut Storage Costs

Reduce storage costs by forwarding [compact](https://doc.log10x.com/run/transform/#compact) events to object storage (e.g., AWS S3, Azure Blobs). Expand events on-the-fly using the [Retriever](../../retriever/ "Stream events from low-cost storage to log analyzer and metric outputs on-demand") app to forward raw data to log analytics and metric outputs periodically or on-demand.

Cut Log Analytics Costs

Reduce ingestion and licensing costs, where the destination supports it, by forwarding [compact](https://doc.log10x.com/run/transform/#compact) events to Splunk or self-hosted Elasticsearch/OpenSearch. The open-source [10x for Splunk](splunk.md) app and [L1ES Elasticsearch plugin](elasticsearch.md) expand events in real-time, maintaining full querying, dashboard, and alerting capabilities; compact is lossless, so expanded events match the originals. ## :material-cog-transfer-outline: Workflow Compact mode processes events from a variety of [log forwarders](https://doc.log10x.com/run/input/forwarder), such as Fluentd, Fluent Bit, Filebeat, and Logstash. Configure the app to process all or a subset of the events, allowing for targeted analysis and volume reduction.
```mermaid graph LR A["
🚙 Forwarder
Sidecar Process
"] --> B["
📡 Receive
Stream Events
"] B --> C["
🔄 Transform
into TenXObjects
"] C --> D["
🎁 Enrich
Add Context
"] D --> E["
🗜️ Compact
Encode Events
"] E --> F["
📤 Output
Return to Forwarder
"] classDef deploy fill:#7c3aed88,stroke:#6d28d9,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef receive fill:#9333ea88,stroke:#7c3aed,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef transform fill:#2563eb88,stroke:#1d4ed8,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef enrich fill:#059669,stroke:#047857,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef compact fill:#f59e0b,stroke:#d97706,color:#ffffff,stroke-width:2px,rx:8,ry:8 classDef output fill:#ea580c88,stroke:#c2410c,color:#ffffff,stroke-width:2px,rx:8,ry:8 class A deploy class B receive class C transform class D enrich class E compact class F output ```
🚙 **Forwarder**: Runs 10x as a [sidecar process](https://doc.log10x.com/engine/launcher/sidecar) to log forwarders for real-time event analysis 📡 **Receive**: Read events continuously from [log forwarders](https://doc.log10x.com/run/input/forwarder) via IPC 🔄 **Transform**: Structure raw events into well-defined [TenXObjects](https://doc.log10x.com/run/transform/) 🎁 **Enrich**: Apply [enrichment rules](https://doc.log10x.com/run/initialize/) to augment TenXObjects with intelligent context 🚦 **Filter**: Shed over-budget events via [per-node budget sampling or a declarative field-set mute file](https://doc.log10x.com/run/receive/rate/) to prevent over-billing 🗜️ **Compact**: [Losslessly compact](https://doc.log10x.com/run/transform/#compact) events using templates the runtime engine builds from [AOT compiler](https://doc.log10x.com/compile/) symbols 📤 **Output**: Return compact events to forwarder to ship to destination analyzers or storage ## :material-hexagon-multiple-outline: Architecture Compact mode executes inside the [Receiver](../index.md) sidecar, compacting events ***before*** they ship to a log analyzer or object storage. === ":octicons-x-12: No 10x" Forwarders ship verbose events containing repetitive elements such as JSON/KV field names, messages, severity levels and app-specific low-cardinality values, inflating cost by a large margin, the share that compact recovers where the destination supports it.
![Architecture diagram: Log forwarders ship verbose events with repetitive JSON field names and low-cardinality values to log analyzers, inflating log-analyzer cost by the share that compact recovers](../../../assets/tenx-edge-optimizer-none.png){ align=left }
:x: Forwarders ship costly duplicative events to log analyzers
=== ":material-aws: Storage" Forward compact events to low-cost storage (e.g., AWS S3, Azure Blobs). The [Retriever](../../retriever/) app expands and streams selected events to log analyzers and metric outputs.
![Architecture diagram: Compact mode losslessly compacts events before uploading to S3 or S3-compatible storage, where Retriever expands and streams them on demand](../../../assets/tenx-edge-optimizer-storage.png){ align=left }
:white_check_mark: **Compact mode** losslessly compacts events before they upload to storage
=== ":material-greater-than: Splunk" Forward compact events to Splunk. The open-source [10x for Splunk](splunk.md) app expands events on-the-fly at search time, displaying them in full JSON/text form in dashboards and queries.
![Architecture diagram: Compact mode losslessly compacts events before shipping to Splunk, where the 10x for Splunk app expands them at search time](../../../assets/tenx-edge-optimizer-splunk.png){ align=left }
:white_check_mark: **Compact mode** losslessly compacts events before they ship to Splunk
=== ":simple-elasticsearch: Elasticsearch" Forward compact events to Elasticsearch or OpenSearch. The open-source [L1ES plugin](elasticsearch.md) transparently rewrites standard queries and decodes `_source` at search time, Kibana dashboards, saved searches, and alerts work unchanged.
![Architecture diagram: Compact mode losslessly compacts events before shipping to Elasticsearch or OpenSearch, where the L1ES plugin expands them at search time](../../../assets/tenx-edge-optimizer-elasticsearch.png){ align=left }
:white_check_mark: **Compact mode** losslessly compacts events before they ship to ElasticSearch
## :material-shield-check-outline: Safety & Reliability Compact mode runs inside the Receiver sidecar with fail-open design, if the receiver crashes or stops, your logs continue flowing normally at full volume to your analyzer. |Topic|Detail| |---|---| |[Fail-open design](../faq.md#what-happens-if-the-sidecar-fails)|Logs continue flowing if 10x goes down| |[Backpressure handling](../faq.md#how-does-the-receiver-handle-failures)|Disk buffering prevents data loss during spikes| |[Resource requirements](../faq.md#which-log-forwarders-does-the-receiver-support)|512MB heap + 2 threads handles 100+ GB/day| |[Rollback](../faq.md#what-happens-if-the-sidecar-fails)|`helm uninstall` takes ~1 minute, no data loss| See the [Receiver FAQ](../faq.md) for complete operational details, capacity planning, and deployment guidance.