--- title: "Delete Index Objects" description: "Delete index objects for target blob" source: "https://github.com/log-10x/modules/tree/main/pipelines/run/modules/input/objectStorage/delete/module.yaml" icon: "material/target" hidden: "true" --- Deletes [TenXTemplate Filters](https://doc.log10x.com/run/input/objectStorage/index/#tenxtemplate-filters) when a storage object (e.g., S3) is removed. Runs as a deletion trigger to clean up associated index objects. ## :material-wrench-outline: Config Files To configure the Delete index objects module, [:material-cog: Edit](https://doc.log10x.com/config/app/#module-config "Learn how to edit app and module configurations") these files. Below is the default configuration from: [delete/config.yaml](https://github.dev/log-10x/config/blob/main/pipelines/run/input/objectStorage/delete/config.yaml "delete/config.yaml"){target="\_blank"}.
Edit Online
Edit config.yaml Locally
```yaml # 🔟❎ 'run' Object storage delete index objects configuration # Configure query AWS S3 bucket settings # To learn more see https://doc.log10x.com/run/input/objectStorage/index/delete # Set the 10x pipeline to 'run' tenx: run # =============================== Dependencies ================================ include: run/modules/input/objectStorage/index/delete # ============================= AWS S3 Query Options ========================== indexDelete: # 'objectStorageName' selects the storage accessor alias registered by 'objectStorageClassName' (AWS or Azure). # Set TENX_OBJECT_STORAGE_NAME to Azure to target Azure Blob Storage. Defaults to AWS. # To learn more about configuring Object storage, see: https://doc.log10x.com/run/input/objectStorage # AWS Authentication is made via the default provider credentials chain: # https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html # AWS Region selection is made via the default region provider chain: # https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/regions/providers/DefaultAwsRegionProviderChain.html - objectStorageName: $=TenXEnv.get("TENX_OBJECT_STORAGE_NAME", "AWS") # 'objectStorageArgs' provides arguments to the constructor of the class defined by 'objectStorageName'. # The 'invoke' argument sets how the storage accessor routes pipeline invocations. # AWS accepts sqs, lambda or http. Azure accepts queue (Storage Queue) or http. # Set TENX_OBJECT_STORAGE_INVOKE to override the default of sqs. An unknown value fails the accessor at construction. objectStorageArgs: - invoke - $=TenXEnv.get("TENX_OBJECT_STORAGE_INVOKE", "sqs") # 'container' specifies the container in which index objects created by the 'index' module reside. # On AWS the S3 event notification in TENX_LAMBDA_INPUT carries the bucket name at Records[0].s3.bucket.name. # On Azure the Event Grid BlobCreated event in TENX_LAMBDA_INPUT carries the blob URL at data.url as # https://{account}.blob.core.windows.net/{container}/{blob}. The container is the path segment after the host: # the first '/' found from index 8 closes the host (index 8 skips the scheme), the next '/' closes the container. container: '$=TenXEnv.get("TENX_OBJECT_STORAGE_NAME", "AWS") == "Azure" ? TenXString.substring(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), TenXString.indexOf(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), "/", 8) + 1, TenXString.indexOf(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), "/", TenXString.indexOf(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), "/", 8) + 1)) : TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.Records[0].s3.bucket.name")' # 'target' sets a logical name under which output index objects reside. # This commonly refers to the name of the application/service which generated # 'targetObject' (e.g., 'my-service', 'my-broker') target: app # 'targetObject' points to the blob (e.g., log file) within being deleted # On AWS this is Records[0].s3.object.key. On Azure it is the rest of data.url after the container segment. object: '$=TenXEnv.get("TENX_OBJECT_STORAGE_NAME", "AWS") == "Azure" ? TenXString.substring(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), TenXString.indexOf(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), "/", TenXString.indexOf(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"), "/", 8) + 1) + 1, TenXString.length(TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.data.url"))) : TenXString.jsonpath(TenXEnv.get("TENX_LAMBDA_INPUT"), "$.Records[0].s3.object.key")' ``` ## :material-menu: Options Specify the options below to [configure](/config "configure") multiple Delete index objects: |Name|Description|Category| |---|---|---| |[indexDeleteObjectStorageName](#indexdeleteobjectstoragename "object storage logical name")|Object storage logical name|General| |[indexDeleteContainer](#indexdeletecontainer "Object storage container containing index objects")|Object storage container containing index objects|General| |[indexDeleteObject](#indexdeleteobject "the target object whose index objects to delete from storage")|The target object whose index objects to delete from storage|General| |[indexDeleteObjectStorageArgs](#indexdeleteobjectstorageargs "custom object storage args")|Custom object storage args|Advanced| |[indexDeleteTarget](#indexdeletetarget "logical name identifying the origin of 'indexDeleteObject'")|Logical name identifying the origin of 'indexDeleteObject'|Output| ### General #### :material-menu-right-outline:**`indexDeleteObjectStorageName`** Object storage logical name. |Type|Required|Category| |---|---|---| |String|✔|General| Identifies the [object storage](https://doc.log10x.com/run/input/objectStorage#objectstorageccessclassname) from which to delete the target index objects (e.g., 'AWS'). #### :material-menu-right-outline:**`indexDeleteContainer`** Object storage container containing index objects. |Type|Required|Category| |---|---|---| |String|✔|General| Specifies the object storage container (e.g., AWS S3 bucket) name from which to delete [TenXTemplate Filters](https://doc.log10x.com/run/input/objectStorage/index/#tenxtemplate-filters). #### :material-menu-right-outline:**`indexDeleteObject`** The target object whose index objects to delete from storage. |Type|Required|Category| |---|---|---| |String|✔|General| Sets the name of the object (e.g., log file) whose associated index objects to delete from [indexDeleteContainer](https://doc.log10x.com/run/input/objectStorage/index/delete/#indexdeletecontainer). ### Advanced #### :material-menu-right-outline:**`indexDeleteObjectStorageArgs`** Custom object storage args. |Type|Default|Category| |---|---|---| |List|\[\]|Advanced| Custom arguments passed as a map to the constructor of the underlying [object storage](https://doc.log10x.com/run/input/objectStorage/index/delete/#queryobjectobjectstoragename). This list is expected to hold pairs of key values (e.g., args: \[key1, value1, key2, value2\]). ### Output #### :material-menu-right-outline:**`indexDeleteTarget`** Logical name identifying the origin of 'indexDeleteObject'. |Type|Required|Category| |---|---|---| |String|✔|Output| Specifies a logical name to delete [TenXTemplate Filters](https://doc.log10x.com/run/input/objectStorage/index/#TenXTemplate-filters) produced for [indexDeleteTargetObject](https://doc.log10x.com/run/input/objectStorage/index/delete/#indexdeletetargetObject) under. This name commonly specifies the app which generated the events enclosed within this blob (e.g. 'acme-client').
:material-github: This module is defined in [delete/module.yaml](https://github.com/log-10x/modules/tree/main/pipelines/run/modules/input/objectStorage/delete/module.yaml "delete/module.yaml"){target="\_blank"}.