# Lumenare Search

Advanced WordPress search plugin with instant live search, predictive keywords, and filterable results.

## Features

- **Instant Live Search**: Real-time search results as users type, with optional featured-image thumbnails
- **Guided Setup Presets**: Recommended presets for publishers, commerce sites, knowledge bases, high-volume installs, and contextual ASP migrations
- **Modular Suggestions**: Provider-based suggestions from analytics, imported migration stats, titles, taxonomies, and index terms
- **Custom Search Index**: Fast database-driven search using weighted keywords
- **Filterable Results**: Filter by content type, categories, tags, custom taxonomies, and date ranges
- **Search Intelligence**: Settings-tab reporting for CTR metrics, result-level opportunity actions, query promotions, synonym drafts, remediation outcomes, Search ROI summaries, zero-result opportunities, low-click searches, and import history
- **Trending and Latest Search Lists**: Configurable public widget and block surfaces for analytics-backed search trends
- **Ajax Search Pro Migration**: When ASP data is detected, migrate historical analytics, impressions, clicks, and suggestion terms from the Search Imports tab, then clean up old ASP database leftovers
- **Multiple Integration Points**: 
  - Replaces default WordPress search
  - Shortcode: `[lumenare_search]`
  - Search and search-trends widgets for sidebars
  - Native Gutenberg blocks for search forms, filters, and search trends
- **Secure & Standards Compliant**:
  - Follows WordPress Plugin Coding Standards
  - PSR-4 autoloading
  - Proper input sanitization and output escaping
  - Prepared SQL statements
  - Nonce verification for all AJAX requests

## Installation

1. Upload the `lumenare-search` folder to `/wp-content/plugins/`
2. Run `composer dump-autoload` in the plugin directory
3. Activate the plugin through the WordPress admin panel
4. Go to Settings > Lumenare Search to configure

## Configuration

### General Settings
- Enable/disable the plugin
- Set search input placeholder text
- Configure minimum characters before search begins
- Tune the live search trigger delay for instant results vs. server load
- **Keyword Match Mode** - Choose between matching ANY keyword (OR) or ALL keywords (AND)
- **Customize stop words** - Define which common words to exclude from indexing for better search relevance
- **Configure synonyms** - Define equivalent terms to expand search coverage and improve recall

### Advanced Search Features
- **Fuzzy Matching (Typo Tolerance)** - Find results even with misspellings using Levenshtein distance
- **Configurable Threshold** - Control how strict or permissive fuzzy matching should be
- **Autocorrect Prevention** - Disable browser/device autocorrect to preserve technical terms and brand names
- **Search Term Highlighting** - Visually highlights matching terms in titles and excerpts with context-aware placement

### Content Types
- Select which post types to include in search results
- Supports posts, pages, and custom post types

### Search Filters
- Enable/disable taxonomy facets and content-type filtering
- Choose which public taxonomies appear as facets
- Limit how many facet values appear per group
- Enable/disable date range filtering

### Display Settings
- Results per page on full results page
- Number of results in live dropdown
- Optional featured images in live dropdown results

## Usage

### Default Search Replacement
The plugin automatically replaces the default WordPress search form site-wide, including:
- Traditional search forms via `get_search_form()`
- **Gutenberg Search blocks** - Any Search block added in the block editor
- Theme search forms

### Gutenberg/Block Editor
When you add a **Search block** in the Gutenberg editor, it will automatically use Lumenare Search with live results. Lumenare also includes native blocks for the search form, search filters, and trending/latest search lists.

### Shortcode
Use the shortcode in posts, pages, or templates:

```php
[lumenare_search]
```

With custom placeholder:
```php
[lumenare_search placeholder="Search our site..."]
```

### Widget
Add the "Lumenare Search" widget to any widget area through Appearance > Widgets.

### Trending And Latest Search Lists
Lumenare Search can display public lists of trending or latest successful searches through the **Lumenare Search Trends** widget or the **Lumenare Search Trends** block. Both surfaces use the same analytics-backed data source and can be configured for list or chip layouts, result count visibility, period, item limit, and zero-result filtering.

Latest searches default to repeated successful queries so one-off private searches are not promoted publicly. Lower the minimum occurrence setting only when the site's analytics policy allows raw latest-query display.

### Template Function
Use in theme templates:

```php
<?php
if ( function_exists( 'get_search_form' ) ) {
    get_search_form();
}
?>
```

## Keyword Match Mode

Control how multiple search keywords are matched to balance precision vs. recall.

### ANY Mode (OR Logic) - Default
**When to use:** Broad topic searches, exploratory queries, maximizing results

Returns results containing **at least one** of the search keywords.

**Example:** Search "Sega Master System"
- Finds posts with "Sega" OR "Master" OR "System"
- Result: 50+ results including Sega Genesis, Master System, Atari System
- **Pro:** More results, better for discovery
- **Con:** May include less relevant results

### ALL Mode (AND Logic) - Precision
**When to use:** Product names, specific phrases, technical queries, reducing noise

Returns **only** results containing **ALL** search keywords.

**Example:** Search "Sega Master System"
- Finds only posts with "Sega" AND "Master" AND "System"
- Result: 8 results, all about Sega Master System specifically
- **Pro:** Highly relevant, precise results
- **Con:** Fewer results, may miss relevant content with different wording

### Which Mode to Choose?

**Use ANY (OR) for:**
- General topic searches ("video games history")
- Blog content discovery
- When you want maximum coverage
- Research and exploration

**Use ALL (AND) for:**
- Product names ("Nintendo Game Boy")
- Technical terms ("database query optimization")
- Specific phrases ("civil war battlefield")
- E-commerce product searches
- When precision matters more than quantity

### Configuration

1. Go to Settings > Lumenare Search
2. Find "Keyword Match Mode" in General Settings
3. Select your preferred mode:
   - **Match ANY keyword (OR)** - Default, broader results
   - **Match ALL keywords (AND)** - Precise, fewer results
4. Click "Save Settings"
5. Changes apply immediately (no reindexing needed)

### Technical Details

- **ANY mode:** Uses SQL `OR` - fast, efficient
- **ALL mode:** Uses `HAVING` clause to ensure all keywords present
- Works seamlessly with synonyms and fuzzy matching
- Original search keywords counted before synonym/fuzzy expansion
- Both modes maintain relevance scoring (weighted by title/excerpt/content)

## Search Index

The plugin creates a custom database table to store keyword indexes for fast searching. Keywords are extracted from:
- Post titles (weight: 3)
- Post excerpts (weight: 2)
- Post content (weight: 1)

### Stop Words

Stop words are common words that are excluded from search indexing to improve relevance and performance. Examples include "a", "an", "the", "is", "are", etc.

**Why Stop Words Matter:**
- Reduces index size and improves search speed
- Eliminates noise from common words that don't add meaning
- Focuses search results on meaningful content words
- Improves relevance by preventing matches on trivial words

**Customizing Stop Words:**
1. Go to Settings > Lumenare Search
2. Scroll to the "Stop Words" field in General Settings
3. Add or remove words (one per line)
4. Click "Save Settings"
5. **Important:** Click "Reindex All Content" to apply changes to existing content

**Default Stop Words Included:**
The plugin includes a comprehensive English stop word list covering common articles, pronouns, prepositions, and conjunctions.

### Synonyms

Synonyms are groups of equivalent terms that expand search coverage. When a user searches for one word, results containing any of its synonyms will also be found.

**Why Synonyms Matter:**
- Improves search recall - finds more relevant results even with different terminology
- Handles user vocabulary variations (e.g., "car" vs "automobile")
- Supports industry-specific jargon and common abbreviations
- Works at query time - no reindexing needed
- Bidirectional - all words in a group are treated equally

**How Synonyms Work:**
When you define `car, automobile, vehicle` as synonyms:
- Search for "car" → finds results with car, automobile, OR vehicle
- Search for "automobile" → finds results with car, automobile, OR vehicle
- Search for "red car" → finds "red automobile" and "red vehicle" too

**Configuring Synonyms:**
1. Go to Settings > Lumenare Search
2. Scroll to the "Synonyms" field in General Settings
3. Add synonym groups (comma-separated, one group per line):
   ```
   car, automobile, vehicle
   doctor, physician, dr
   quick, fast, rapid, speedy
   buy, purchase, order
   help, assist, support
   ```
4. Click "Save Settings"
5. **No reindexing needed!** Synonyms work instantly at search time

**Best Practices:**
- Group truly equivalent terms together
- Include common abbreviations and acronyms
- Add industry-specific terminology relevant to your content
- Keep groups focused - don't add loosely related words
- Test searches after adding synonyms to verify behavior

**Example Use Cases:**
- **E-commerce**: "buy, purchase, order" or "shirt, top, blouse"
- **Healthcare**: "doctor, physician, dr" or "medicine, medication, drug"
- **Technology**: "computer, pc, laptop" or "software, program, app"
- **Education**: "course, class, lesson" or "student, learner, pupil"

**Default Synonyms Included:**
The plugin includes common English synonym groups covering actions like buy/purchase, help/assist, quick/fast, etc.

### Fuzzy Matching (Typo Tolerance)

Fuzzy matching allows users to find results even when they misspell words. It uses the Levenshtein distance algorithm to measure similarity between the search query and indexed keywords.

**Why Fuzzy Matching Matters:**
- Users make typos, especially on mobile devices
- Improves user experience - finds what they meant, not just what they typed
- Handles common misspellings automatically
- Reduces "no results" frustration

**How It Works:**
The system calculates the "edit distance" between the misspelled word and indexed keywords. Edit distance is the number of single-character changes needed to transform one word into another.

**Examples (with threshold 2):**
- "computr" → "computer" (distance: 1, found!)
- "searchh" → "search" (distance: 1, found!)
- "qwick" → "quick" (distance: 1, found!)
- "recieve" → "receive" (distance: 2, found!)
- "buisness" → "business" (distance: 2, found!)

**Configuring Fuzzy Matching:**
1. Go to Settings > Lumenare Search
2. Scroll to "Advanced Search Features"
3. Check "Enable typo-tolerant search (fuzzy matching)"
4. Set the threshold (recommended: 2):
   - **1**: Very strict - only single-character typos
   - **2**: Balanced - handles most common typos (default)
   - **3**: Permissive - may return less relevant results
   - **4-5**: Very permissive - use with caution
5. Click "Save Settings"

**Performance Considerations:**
- Only applies to words 4+ characters (short words excluded)
- Checks limited set of similar-length keywords
- Uses PHP's optimized built-in `levenshtein()` function
- May add slight delay on large indexes - test with your content
- Disabled by default - enable when needed

**Best Practices:**
- Start with threshold 2 and adjust based on your results
- Test with common misspellings in your niche
- Monitor search performance with large indexes
- Consider disabling if you have a very large site (100k+ posts)

### Autocorrect Prevention

Browser and device autocorrect can interfere with search queries by "fixing" technical terms, brand names, and specific keywords that users intentionally type.

**The Problem:**
- User searches for "Qdrant" (a database name)
- Autocorrect changes it to "Quadrant"
- Search returns wrong results or no results
- User frustration increases

**The Solution:**
When enabled, adds HTML attributes to the search input that prevent browsers and devices from modifying user input:
- `autocorrect="off"` - Disables iOS autocorrect
- `autocomplete="off"` - Disables browser autocomplete suggestions
- `autocapitalize="off"` - Prevents automatic capitalization
- `spellcheck="false"` - Disables spell checking underlines

**When to Enable (Default: ON):**
- **Technical/SaaS sites** - Product names, APIs, tech terms
- **E-commerce** - Brand names, model numbers, SKUs
- **Healthcare** - Medical terms, drug names
- **Education** - Course codes, specific terminology
- **Any site** where precise keyword matching matters

**When to Disable:**
- General consumer blogs where autocorrect helps users
- Sites where most searches are common English words
- When you want to help users with basic spelling

**Configuring Autocorrect:**
1. Go to Settings > Lumenare Search
2. Scroll to "Advanced Search Features"
3. Check/uncheck "Disable browser/device autocorrect in search input"
4. Click "Save Settings"
5. Changes apply immediately (no reindexing needed)

**Complementary Features:**
- Use **Autocorrect Prevention** to preserve user input exactly as typed
- Use **Fuzzy Matching** to handle actual typos and misspellings
- Together they give users control while still being forgiving

### Search Term Highlighting

Search term highlighting visually marks matching keywords in search results to show users WHY results matched their query.

**Why Highlighting Matters:**
- **Immediate visual feedback** - Users instantly see why results matched
- **Validates relevance** - Confirms search is working correctly
- **Faster result scanning** - Users can quickly identify the most relevant results
- **Context awareness** - Shows matching terms in surrounding text
- **Improved UX** - Professional search experience similar to Google, Elasticsearch, etc.

**What Gets Highlighted:**
- Original search terms entered by the user
- **Synonym matches** - If synonyms are configured and used
- **Fuzzy matches** - If fuzzy matching finds similar terms
- All highlighting respects word boundaries for clean display

**How It Works:**
1. User searches for "fast computer"
2. Results show with **fast** and **computer** highlighted in yellow
3. Excerpts are context-aware - they show text AROUND the match, not just the beginning
4. If synonyms are configured (e.g., "computer, pc, laptop"), all variations are highlighted

**Context-Aware Excerpts:**
Instead of always showing the first 30 words, the system intelligently extracts text surrounding the matched keywords. This means users see the most relevant part of the content, not just the introduction.

**Example:**
Search: "database performance"
Standard excerpt: "Welcome to our blog. Today we're discussing various topics..."
**Context-aware excerpt:** "...optimize **database** indexes for better query **performance**. This significantly improves..."

**Configuring Highlighting:**
1. Go to Settings > Lumenare Search
2. Scroll to "Advanced Search Features"
3. Check/uncheck "Enable Search Term Highlighting"
4. Click "Save Settings"

**Customizing Styles:**
Highlighted terms use the CSS class `lumenare-highlight` with default yellow background. Customize in your theme's CSS:

```css
.lumenare-highlight,
mark.lumenare-highlight {
    background-color: #your-color;
    color: #text-color;
    font-weight: bold;
    padding: 0 2px;
    border-radius: 2px;
}
```

**Performance:**
- Highlighting happens during search query processing
- Uses efficient regex matching with word boundaries
- Properly escapes all output to prevent XSS
- Cached with search results (5-minute transient)
- No performance impact when disabled

**Best Practices:**
- Keep enabled (default) for best user experience
- Works perfectly with synonyms and fuzzy matching
- Highlighting is subtle but effective - users notice it subconsciously
- Yellow background is proven to be highly visible without being distracting

### Reindexing
Go to Settings > Lumenare Search and click "Reindex All Content" to rebuild the search index. This is useful after:
- Initial plugin activation
- Bulk importing content
- Changing which post types to search
- **Modifying stop words**
- Updating search relevance

## Technical Details

### Architecture
- **PSR-4 Autoloading**: All classes follow PSR-4 standards
- **Namespaces**: `LumenareSearch\*`
- **Database**: Custom table `wp_lumenare_search_index` with fulltext indexes
- **Caching**: Search results cached in transients (5 minutes)
- **Stop Words**: Applied during indexing to reduce noise
- **Synonyms**: Applied during query processing to expand coverage
- **Fuzzy Matching**: Optional Levenshtein distance algorithm for typo tolerance
- **Autocorrect Prevention**: HTML attributes to preserve exact user input
- **Highlighting**: Context-aware excerpt extraction with regex-based term marking

### Security
- All inputs sanitized with `sanitize_text_field()`, `absint()`, etc.
- All outputs escaped with `esc_html()`, `esc_url()`, `esc_attr()`
- Database queries use `$wpdb->prepare()` exclusively
- AJAX requests protected with nonce verification
- User capability checks on admin actions

### Hooks & Filters

#### Actions
- `save_post` - Indexes content when posts are saved
- `delete_post` - Removes content from index when deleted
- `wp_ajax_lumenare_search` - Handles live search AJAX requests
- `wp_ajax_nopriv_lumenare_search` - Handles live search for logged-out users

#### Filters
- `get_search_form` - Replaces default search form
- `render_block` - Replaces Gutenberg Search block output
- `template_include` - Loads custom search results template
- `pre_get_posts` - Modifies search query with filters

## Requirements

- PHP 7.4 or higher
- WordPress 6.0 or higher
- MySQL 5.6 or higher (for FULLTEXT indexes)

## File Structure

```
lumenare-search/
├── src/                      # PSR-4 classes
│   ├── Core/                 # Core plugin functionality
│   ├── Search/               # Search and indexing
│   ├── Frontend/             # Frontend components
│   └── Admin/                # Admin interface
├── assets/                   # Frontend assets
│   ├── js/                   # JavaScript files
│   └── css/                  # Stylesheets
├── admin/                    # Admin assets
│   ├── js/
│   └── css/
├── templates/                # Template files
├── vendor/                   # Composer autoloader
├── lumenare-search.php     # Main plugin file
├── uninstall.php            # Uninstall cleanup
└── composer.json            # Composer configuration
```

## License

GPL v2 or later

## Author

Adam Greenwell

## Changelog

### 2.1.02
- Fixed automatic new-content reindex scheduling so stale overdue WP-Cron events are repaired and rescheduled
- Improved WP-Cron schedule labels so bootstrap does not trigger early translation notices

### 2.1.01
- Added a configurable Lumenare Search Trends widget for trending or latest public search lists
- Added a native dynamic Gutenberg block for trending/latest search lists, built with block metadata and server-side rendering
- Added privacy-minded defaults for latest searches so one-off zero-result queries are not displayed publicly
- Added opt-in featured-image thumbnails for live search dropdown results
- Added image-aware AJAX result payloads and cache separation between thumbnail and text-only live responses
- Updated live dropdown styling so image results stay compact across desktop and mobile layouts
- Improved ListenUp audio boost settings so controls appear only when ListenUp is installed, with a contextual WordPress.org install prompt

### 2.1.00
- Added automatic new-content partial reindexing with configurable WP-Cron intervals, run status, and safe continuation handling for large catalogs
- Added guided setup presets for publisher, commerce, knowledge base, high-volume, and Ajax Search Pro migration sites
- Added Search Intelligence, Search ROI, reviewed/reopen state, exact-query promotions, editable synonym drafts, and remediation outcome tracking
- Added Ajax Search Pro migration tooling for analytics, impressions, clicks, suggestion seeds, contextual import UI, guarded cleanup tools, and `wp lumenare cleanup asp`
- Added native impression tracking, CTR-aware click boosts, source filters, import history, remediation outcomes, and analytics hygiene exclusions
- Added modular suggestion providers, provider order controls, profile-level overrides, short query-focused defaults, and zero-result suggested-search previews
- Added generic taxonomy/content-type/date facets with facet limits plus block and shortcode controls for filter placement
- Added audio attachment ranking boosts with configurable metadata keys and companion-plugin filters
- Improved WordPress 7.0 and PHP 8.3 compatibility, live search trigger delay controls, and safer bootstrap translation behavior
- Improved index table resilience with frontend fallbacks, activation/schema-upgrade repair, reindex-required notices, and hard-error reporting for failed index writes

### 1.1.2
- Added live search results caching - clicking back into search input re-displays cached results without new AJAX request
- Added modern Gutenberg block with 4 variations (Default, Compact, Expanded, Icon Toggle) replacing legacy widget
- Added block editor InspectorControls for configuring search appearance
- Added expandable icon-only search variation for minimal UI footprint
- Fixed fuzzy search now finds misspelled words regardless of starting letter (prioritized candidate retrieval)
- Improved ARIA accessibility attributes for search dropdown elements

### 1.1.1
- Added canonical keyword storage to preserve original capitalization from content
- Improved keyword suggestions with proper spacing and capitalization
- Added URL component filtering to prevent indexing raw URLs.
- Generic word boundary detection that works across different sites and content types
- Note: Reindexing recommended after upgrade to populate canonical keywords and remove URL components

### 1.1.0
- Improve admin ability to fine-tune search results
- Add analytics and search query tracking feature

### 1.0.2
- Improve compatibility with non-block themes

### 1.0.1
- Add option to show search icon
- Strip shortcodes, HTML and other block editor patterns from search index, queries and results


### 1.0.0
- Initial release
- Live search with dropdown results
- Keyword indexing system with weighted relevance (title: 3, excerpt: 2, content: 1)
- Category and date filtering
- Widget and shortcode support
- Admin settings panel with comprehensive options
- **Keyword match mode** - Choose between ANY (OR) or ALL (AND) keyword matching for precision control
- **Configurable stop words** for noise reduction and improved relevance
- **Configurable synonyms** plus Search Intelligence drafts for expanded search coverage and better recall
- **Fuzzy matching (typo tolerance)** with configurable Levenshtein distance threshold
- **Autocorrect prevention** to preserve technical terms and brand names
- **Search term highlighting** with context-aware excerpt extraction
- Gutenberg Search block integration (automatic replacement)
- Transient caching for performance (5-minute cache)
- Full WordPress coding standards compliance
- PSR-4 autoloading architecture
- Secure: nonce verification, input sanitization, output escaping, prepared SQL