--- title: "Defining Queries" description: "Query Console (GUI and CLI), REST API, sample queries, and query parameters for the Retriever" icon: "material/magnify" --- Retriever queries select matching events from storage via a [distributed architecture](https://doc.log10x.com/run/input/objectStorage/query/) that guarantees consistent performance regardless of queried data volume. Log10x is normally driven by an AI agent (Claude, or a model the customer brings) through the log10x MCP server, which installs, configures, and queries via MCP tools. The agent calls `retriever_query`, `retriever_series`, `retriever_query_status`, and `retriever_probe`; install is via `advise_retriever`. The manual steps below are the same operations without an agent. The query handler is served at `POST /streamer/query`. The in-engine app path is `@apps/retriever/query`; the public HTTP route keeps the `/streamer/query` name for now, overridable with `LOG10X_RETRIEVER_QUERY_PATH`. Request body fields are `processingTimeMs`, `resultSizeBytes`, and `name`. The forms `processingTime` / `resultSize` and the `parseDuration` / `parseBytes` sugar are not accepted by the engine. Each query executes in two phases: **1. Scan**: Identifies which files and byte ranges in storage match the target, time range, and search expression. Search expressions support `==`, `&&`, `||`, and `includes()`, and operate on event raw text or fields set by [initializers](https://doc.log10x.com/run/initialize/) during indexing (e.g. `severity_level`, `http_code`, `k8s_namespace`, `country`). **2. Stream**: Fetches matching byte ranges and streams selected events, filtered by the search expression and optional [JavaScript expressions](https://doc.log10x.com/api/js/) for fine-grained control. ## :material-counter: Summaries Stream workers can write per-pattern rollups to a parallel S3 prefix `qrs/{queryId}/{sliceFrom}_{sliceTo}/{worker}.jsonl` instead of (or alongside) raw events at `qr/{queryId}/{sliceFrom}_{sliceTo}/{worker}.jsonl`. Each summary record carries `summaryVolume` (event count), `summaryBytes` (UTF-8 byte total), plus the named enrichment fields the pipeline initialized: typically `severity_level`, `k8s_pod`, `k8s_namespace`, `tenx_user_service`, `message_pattern`. Slice bounds are encoded into the S3 key path. Pick the pathway by query intent: | Pathway | Use when | What's returned | |---|---|---| | Events (`qr/`) | Event lookup, audit-trail retrieval, raw payload retrieval | Full TenXObjects with original text. Truncated when result count exceeds the per-worker cap. | | Summaries (`qrs/`) | Trend, top-pattern, cost-driver tooling at scale | Per-pattern rollups. Counts are exact across all matched events. | Validation on the otel demo: a top-patterns query returned 1,576 events through the events pathway (cap-truncated) and aggregated 7,494 events through summaries. 17 patterns surfaced at ratios 1.80×–8.02× over the events pathway. Toggle at the request: set `writeSummaries: true` on `POST /streamer/query`. The MCP `RetrieverQueryRequest` exposes the same flag and surfaces summaries on the response as `summaries: RetrieverSummary[]`. ## :material-console: Query Console The open source Console executes within your infrastructure and provides a web GUI and CLI for submitting queries via SQS or direct REST, with built-in sample queries, dry run preview, and real-time progress tracking via CloudWatch Logs. ![Screenshot of the Query Console web GUI showing a search expression, drop filters, time range and target selectors, SQS queue submission, and export options](../../assets/stream-console.png)
View on GitHub
=== ":material-web: Web GUI" Start the web console to author and submit queries from a browser: ```bash python3 console.py --serve ``` The GUI runs at `http://localhost:8080` and opens the browser automatically. Use `--port` to change the port, or `--no-browser` to skip the automatic browser launch: ```bash python3 console.py --serve --port 9090 --no-browser ``` === ":material-console-line: CLI" Submit queries directly from the terminal: ```bash # Search for errors in the last 30 minutes python3 console.py \ --search 'severity_level=="ERROR"' --since 30m \ --bucket my-bucket --queue-url $LOG10X_QUERY_QUEUE_URL --follow ``` ```bash # Dry run: print the JSON payload without sending python3 console.py \ --search 'http_code=="500"' --since 1h \ --bucket my-bucket --dry-run ``` ??? tenx-config "CLI options" | Option | Description | |--------|-------------| | `--search EXPR` | Search expression (`==`, `&&`, `\|\|`, `includes()`) | | `--since TIME` | Relative time range (`30s`, `5m`, `1h`, `2d`) | | `--from` / `--to` | Absolute time range in epoch milliseconds | | `--bucket BUCKET` | S3 bucket containing log files | | `--index-bucket BUCKET` | S3 bucket containing index objects (defaults to `--bucket`) | | `--queue-url URL` | SQS queue URL (or set `LOG10X_QUERY_QUEUE_URL` env var) | | `--target PREFIX` | App/service prefix (default: `app`) | | `--processing-time DUR` | CLI-only sugar for max processing time (default: `60s`). The CLI converts it to the `processingTimeMs` REST field. | | `--result-size SIZE` | CLI-only sugar for max result volume (default: `10MB`). The CLI converts it to the `resultSizeBytes` REST field. | | `--log-levels LEVELS` | Comma-separated log levels (default: `ERROR,INFO,PERF`) | | `--filters EXPR` | JavaScript filter expressions for in-memory filtering | | `--no-browser` | Do not open browser automatically when starting web GUI | | `--dry-run` | Print JSON payload without sending | | `--follow` | Poll CloudWatch Logs until query completes | | `--query-id ID` | Monitor an existing query by ID (skips submission, implies `--follow`) | | `--region REGION` | AWS region (default: `us-east-1`) | | `--log-group GROUP` | CloudWatch Logs group (or set `TENX_QUERY_LOG_GROUP`, default: `/log10x/query`) | === ":material-api: REST API" Submit queries directly via HTTP POST to the retriever endpoint: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-5m\")", "to": "now()", "search": "severity_level == \"ERROR\"" }' ``` The query endpoint returns HTTP 200 with a JSON body containing a `queryId` field: ```json { "queryId": "a1b2c3d4-e5f6-7890-abcd-ef1234567890" } ``` Use this ID to track query progress via the [Web GUI](#web-gui) **Monitor** tab or the [CLI](#cli) `--query-id` flag: ```bash python3 console.py --query-id a1b2c3d4-e5f6-7890-abcd-ef1234567890 ``` ??? tenx-requestparams "Request Parameters" | Parameter | Type | Description | |-----------|------|-------------| | `from` | Epoch ms | Start of the time range (inclusive). Use `now("-5m")` for relative time or a literal epoch like `1769076000000`. | | `to` | Epoch ms | End of the time range (exclusive). Use `now()` for current time. | | `search` | String | Search expression using **enriched field names** (e.g., `severity_level`, `http_code`, `k8s_namespace`). Must use enriched fields, NOT raw JSON fields like `stream`, `log`, or `docker`. See [Sample Queries](#sample-queries) for working examples. Supports `&&`, `\|\|`, `==`, and `includes()`. | | `filters` | List | Optional [JavaScript expressions](https://doc.log10x.com/run/input/objectStorage/query/#queryfilters) applied in-memory after the scan phase. Supports all [TenX JavaScript](https://doc.log10x.com/api/js/) functions including `drop()`, `startsWith()`, `endsWith()`, `match()`, and `TenXLookup.get()`. | | `processingTimeMs` | Integer (ms) | Optional max processing time in milliseconds (e.g., `300000` for 5 minutes). Defaults to 60000 (1 minute). | | `resultSizeBytes` | Integer (bytes) | Optional max result volume in bytes (e.g., `10485760` for 10 MB). | | `name` | String | Optional query name. The engine surfaces it as `queryName` in progress logs and summaries. | !!! tenx-safety "Security" Queries can only be initiated by authenticated users with the appropriate AWS IAM permissions. See [security architecture](https://log10x.com/security.html){target="_blank"} for details. ??? tenx-bootstrap "Requirements" - **Python 3.6+** (no external dependencies for the web GUI) - **boto3**: required only for SQS submission and CloudWatch Logs polling. Install with `pip3 install boto3` - **AWS credentials**: configured via `aws configure` or `AWS_ACCESS_KEY_ID`/`AWS_SECRET_ACCESS_KEY` environment variables The console is at `$TENX_MODULES/apps/retriever/console` in your local config, or on GitHub at [retriever/console](https://github.com/log-10x/modules/tree/main/apps/retriever/console){target="_blank"}. ## :material-function-variant: Search expression grammar The search expression is matched against the Bloom index, so a selective expression is dramatically cheaper than an open-ended scan. It uses a small TenX subset: | Operator | Meaning | Example | |---|---|---| | `==` | Field equals a value | `severity_level=="ERROR"` | | `includes(field, "substr")` | Field contains a substring | `includes(text, "ECONNREFUSED")` | | `&&` | Both sides must hold | `severity_level=="ERROR" && includes(text, "timeout")` | | `\|\|` | Either side holds | `k8s_namespace=="prod" \|\| k8s_namespace=="staging"` | Fields are the enrichment fields the pipeline initialized, for example `severity_level`, `k8s_pod`, `k8s_namespace`, `tenx_user_service`, `message_pattern`, plus the raw `text`. Combine them freely, for example `severity_level=="ERROR" && includes(text, "ECONNREFUSED")`. Omit the search to scan the whole window (bounded by the result-size and processing-time limits). For the common case of scoping to a single Reporter-named pattern, pass `pattern` instead of writing a `message_pattern==` expression. ## :material-cloud-search-outline: Sample Queries Ready-to-use query templates for common scenarios. Copy, adjust the time range, and run. ### :material-ambulance: Incident Management ??? tenx-objectstoragequery "Production errors with stack traces" During an outage, fragmented single-line log entries add noise. This query captures every ERROR and CRITICAL event and keeps only [grouped events](https://doc.log10x.com/run/initialize/group/), multi-line stack traces that contain complete failure signatures: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-5m\")", "to": "now()", "search": "severity_level == \"ERROR\" || severity_level == \"CRITICAL\"", "filters": ["drop(!isGroup)"] }' ``` ??? tenx-objectstoragequery "HTTP 500 errors in a namespace, excluding health checks" Isolate server errors in a specific Kubernetes namespace while filtering out noise from health and readiness probes. The scan phase narrows by [http_code](https://doc.log10x.com/run/initialize/httpCode/) and [k8s_namespace](https://doc.log10x.com/run/initialize/k8s/), then `endsWith()` removes probe endpoints: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-15m\")", "to": "now()", "search": "http_code == \"500\" && k8s_namespace == \"production\"", "filters": ["drop(endsWith(\"/health\"))", "drop(endsWith(\"/ready\"))"] }' ``` ??? tenx-objectstoragequery "Errors by source code origin" After a deployment, narrow errors to a specific container and code module. The [symbol_origin](https://doc.log10x.com/run/initialize/message/) field identifies the source file that produced each event, letting you pinpoint regressions to specific code paths: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-15m\")", "to": "now()", "search": "severity_level == \"ERROR\" && k8s_container == \"order-service\"", "filters": ["drop(!includes(symbol_origin, \"Checkout\"))"] }' ``` ??? tenx-objectstoragequery "Deduplicate repeated stack traces" A single crashing code path can emit thousands of identical stack traces per minute. This query uses [TenXCounter](https://doc.log10x.com/api/js/#TenXCounter.inc) to pass only the first 3 occurrences of each unique [message_pattern](https://doc.log10x.com/run/initialize/message/) per 5-minute window: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-30m\")", "to": "now()", "search": "severity_level == \"CRITICAL\" && isGroup == true", "filters": ["drop(TenXCounter.inc(concat(\"trace_\", message_pattern), 1, \"5m\") > 3)"] }' ``` ### :material-shield-check: Auditing & Compliance ??? tenx-objectstoragequery "Failed access attempts with geographic context" SOC 2 and ISO 27001 require logging unauthorized access attempts. This query surfaces all HTTP 401/403 responses, automatically enriched with the originating [country](https://doc.log10x.com/run/initialize/geoIP/) via GeoIP lookup on the source IP address: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-24h\")", "to": "now()", "search": "http_code == \"401\" || http_code == \"403\"" }' ``` ??? tenx-objectstoragequery "Admin actions from unexpected regions" Surface admin-level events originating from outside expected geographies. The `drop()` filter inverts an allow-list, dropping events from known [countries](https://doc.log10x.com/run/initialize/geoIP/) so only anomalous origins remain: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-24h\")", "to": "now()", "search": "k8s_container == \"admin-api\" && (severity_level == \"WARN\" || severity_level == \"ERROR\")", "filters": ["drop(country == \"US\" || country == \"DE\" || country == \"GB\")"] }' ``` ??? tenx-objectstoragequery "Configuration changes outside approved windows" Change management requires every modification to be traceable to an approved window. This query finds config-related events in the infra namespace, then cross-references each pod against a [lookup table](https://doc.log10x.com/api/js/#TenXLookup.get) of approved changes, dropping approved events so only unauthorized modifications surface: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-48h\")", "to": "now()", "search": "k8s_namespace == \"infra\" && includes(text, \"config\")", "filters": ["drop(TenXLookup.get(\"change_windows\", k8s_pod, \"pod\", \"approved\") == \"true\")"] }' ``` ### :material-lock-alert: Information Security ??? tenx-objectstoragequery "Brute force detection by country" Credential stuffing attacks generate high volumes of authentication failures from narrow geographic origins. This query uses [TenXCounter](https://doc.log10x.com/api/js/#TenXCounter.inc) to count HTTP 401 events per [country](https://doc.log10x.com/run/initialize/geoIP/) in 3-minute rolling windows, surfacing only countries exceeding 50 failures: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-1h\")", "to": "now()", "search": "http_code == \"401\"", "filters": ["drop(TenXCounter.inc(concat(\"authfail_\", country), 1, \"3m\") < 50)"] }' ``` ??? tenx-objectstoragequery "SQL injection pattern detection" Scan HTTP 400 and 500 responses for common SQL injection patterns using [regex matching](https://doc.log10x.com/api/js/#TenXString.match). Events not matching the pattern are dropped, leaving only those containing suspicious SQL keywords: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-12h\")", "to": "now()", "search": "http_code == \"400\" || http_code == \"500\"", "filters": ["drop(!match(text, \"(?i)(SELECT|UNION|DROP|INSERT|DELETE).*(FROM|WHERE|TABLE)\"))"] }' ``` ??? tenx-objectstoragequery "Geographic anomalies in production access" Threat hunting for access from outside expected regions. This query finds successful requests to the production namespace and drops events from North America and Europe, surfacing access from any other [continent](https://doc.log10x.com/run/initialize/geoIP/) for investigation: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-6h\")", "to": "now()", "search": "k8s_namespace == \"production\" && http_code == \"200\"", "filters": ["drop(continent == \"NA\" || continent == \"EU\")"] }' ``` ??? tenx-objectstoragequery "Large response payload detection" Data exfiltration often manifests as requests returning abnormally large responses. This query targets a sensitive container, uses [regex](https://doc.log10x.com/api/js/#TenXString.match) to extract the response size, and [parses it](https://doc.log10x.com/api/js/#TenXMath.parseInt) to keep only responses exceeding 50 MB: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-4h\")", "to": "now()", "search": "k8s_container == \"reporting-api\" && http_code == \"200\"", "filters": ["drop(TenXMath.parseInt(match(text, \"bytes_sent=([0-9]+)\")) < 52428800)"] }' ``` ### :material-chart-bar: Metric Aggregation ??? tenx-objectstoragequery "Error count by k8s container" Build a live error leaderboard by counting errors per [k8s_container](https://doc.log10x.com/run/initialize/k8s/) in rolling 5-minute windows. [TenXCounter](https://doc.log10x.com/api/js/#TenXCounter.inc) tracks each container independently, and the query surfaces only containers with 10 or more errors: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-1h\")", "to": "now()", "search": "severity_level == \"ERROR\" || severity_level == \"CRITICAL\"", "filters": ["drop(TenXCounter.inc(concat(\"err_\", k8s_container), 1, \"5m\") < 10)"] }' ``` ??? tenx-objectstoragequery "Traffic volume by continent" Aggregate request volume by [continent](https://doc.log10x.com/run/initialize/geoIP/) in hourly windows for capacity planning and CDN configuration. Continents with fewer than 100 requests are dropped, leaving only statistically meaningful traffic segments: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-24h\")", "to": "now()", "search": "http_code == \"200\" || http_code == \"201\"", "filters": ["drop(TenXCounter.inc(concat(\"continent_\", continent), 1, \"1h\") < 100)"] }' ``` ??? tenx-objectstoragequery "Recurring error pattern frequency" Identify which error types dominate by counting each unique [message_pattern](https://doc.log10x.com/run/initialize/message/) in a 30-minute window. Patterns appearing fewer than 25 times are dropped, surfacing only the most frequent error signatures: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-30m\")", "to": "now()", "search": "severity_level == \"ERROR\" || severity_level == \"WARN\"", "filters": ["drop(TenXCounter.inc(concat(\"pat_\", message_pattern), 1, \"30m\") < 25)"] }' ``` ### :material-tune: Dynamic Regulation ??? tenx-objectstoragequery "Drop low-severity noise" During incidents, TRACE and DEBUG events add noise without actionable signal. This filter drops them to focus on events at WARN and above, using the [severity_level](https://doc.log10x.com/run/initialize/level/) field set at index time: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-5m\")", "to": "now()", "search": "severity_level == \"INFO\" || severity_level == \"WARN\" || severity_level == \"ERROR\" || severity_level == \"CRITICAL\"", "filters": ["drop(severity_level == \"TRACE\" || severity_level == \"DEBUG\")"] }' ``` ??? tenx-objectstoragequery "Region-focused monitoring" A regional operations team can restrict query results to a specific [continent](https://doc.log10x.com/run/initialize/geoIP/), dropping all events from other geographies to focus dashboards on local traffic: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-30m\")", "to": "now()", "search": "severity_level == \"ERROR\"", "filters": ["drop(continent != \"EU\")"] }' ``` ??? tenx-objectstoragequery "Health check and readiness probe removal" Kubernetes health and readiness probes generate high-volume INFO events that obscure application-level logs. Multiple `drop()` filters remove these known patterns using `endsWith()` and `includes()` from the [TenX JavaScript API](https://doc.log10x.com/api/js/): ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-30m\")", "to": "now()", "search": "severity_level == \"INFO\"", "filters": ["drop(endsWith(\"/health\"))", "drop(endsWith(\"/ready\"))", "drop(includes(text, \"kube-probe\"))"] }' ``` ??? tenx-objectstoragequery "Stack trace flood control" Rate-limit repeated stack traces during an outage. [TenXCounter](https://doc.log10x.com/api/js/#TenXCounter.inc) tracks each unique combination of [message_pattern](https://doc.log10x.com/run/initialize/message/) and [symbol_origin](https://doc.log10x.com/run/initialize/message/) in 10-minute windows, passing only the first 5 occurrences and dropping the rest: ```bash curl -X POST http://localhost:8080/streamer/query \ -H "Content-Type: application/json" \ -d '{ "from": "now(\"-1h\")", "to": "now()", "search": "isGroup == true && (severity_level == \"ERROR\" || severity_level == \"CRITICAL\")", "filters": ["drop(TenXCounter.inc(concat(\"flood_\", message_pattern, \"_\", symbol_origin), 1, \"10m\") > 5)"] }' ```