| Metric | Official MindAR | TapTapp AR V11 | Improvement |
| :--- | :--- | :--- | :--- |
| **Compilation Time** | ~23.50s | **~1.69s (HD)** | π **~14x Faster** |
| **Output Size (.taar)** | ~770 KB | **~103 KB** | π **86% Smaller** |
| **Matching Logic** | Brute-force | **Nanite LOD (Scale-Filtered)** | π§ **Smart Extraction** |
| **Tracking Data** | 8-bit Gray | **4-bit Packed** | π¦ **50% Data Saving** |
| **Dependency Size** | ~20MB (TFJS) | **< 100KB** | π¦ **99% Smaller Bundle** |
---
## π‘οΈ Robustness & Stability (Stress Tested)
The latest version has been rigorously tested with an adaptive stress test (`robustness-check.js`) covering diverse resolutions (VGA to FHD), rotations (X/Y/Z), and scales.
| Metric | Result | Description |
| :--- | :--- | :--- |
| **Pass Rate** | **96.8%** | High success rate across resolutions (209/216). |
| **Drift Tolerance** | **< 2%** | Validated via sub-pixel coordinate system restoration. |
| **Tracking Precision** | **Double-Precision Fix** | Corrected coordinate scaling for all image octaves. |
| **Detection Time** | **< 10ms** | Ultra-fast initial detection on standard CPU. |
| **Total Pipeline** | **~35ms** | Complete loop (Detect + Match + Track + Validate). |
---
## πΌοΈ Compiler Usage (Automatic / JIT)
The new **TapTapp AR** engine handles compilation automatically in the browser (Just-In-Time). You likely **don't need** to use the offline compiler manually anymore.
Simply pass your image URL to the tracker, and it will compile it on the fly:
```typescript
// No compilation step needed!
const tracker = await startTracking({
targetSrc: './my-image.jpg' // Using a JPG/PNG directly
});
```
However, if you still want to pre-compile for faster startup on low-end devices:
```javascript
import { OfflineCompiler } from '@srsergio/taptapp-ar';
// ... same compiler code as before ...
```
### π¦ Using a Pre-compiled `.taar` File
If you already have a pre-compiled `.taar` file (generated by the offline compiler or provided by another developer), you can skip the JIT compilation entirely by passing it directly to the tracker:
```typescript
// Using a pre-compiled .taar file (skips JIT compilation)
const tracker = await startTracking({
targetSrc: './my-precompiled-target.taar' // Direct .taar file
});
```
**Benefits of using pre-compiled `.taar` files:**
- β‘ **Instant startup**: No compilation delay on first load.
- π± **Better for low-end devices**: Avoids CPU-intensive compilation.
- π **Consistent results**: Same binary across all deployments.
**When to use JIT vs Pre-compiled:**
| Use Case | Recommended Approach |
| :--- | :--- |
| Development / Prototyping | JIT (just use `.jpg`/`.png`) |
| Production / Low-end devices | Pre-compiled `.taar` |
| Dynamic user-uploaded images | JIT |
| Static marketing campaigns | Pre-compiled `.taar` |
---
## π₯ Runtime Usage (AR Tracking)
### 1. The Easy Way: React Component βοΈ
The simplest, zero-config way to add AR to your app:
```tsx
import { TaptappAR } from '@srsergio/taptapp-ar/react';
export const MyARScene = () => (
);
```
### 2. High-Performance React Hook: `useAR` πͺ
For more control over the AR state or custom UI overlays:
```tsx
import { useAR } from '@srsergio/taptapp-ar/react';
const MyCustomAR = () => {
const {
containerRef,
overlayRef,
status,
trackedPoints,
error
} = useAR({
targetImageSrc: "target.png",
scale: 1.0
});
return (
{/* Your custom 3D or 2D overlay */}
{/* Use trackedPoints for custom visualizations */}
{trackedPoints.map((p, i) => (
))}
{/* π This div will be pinned and skewed to the physical target */}
Dynamic UI
);
};
```
#### `TrackingUpdate` Data Object
The `onUpdate` callback provides a rich data object:
- `isTracking`: Boolean status.
- `avgReliability`: (0-1) Confidence of the match.
- `avgStability`: (0-1) Movement smoothness.
- `screenCoords`: Array of `{x, y, id}` of tracked points.
- `worldMatrix`: 4x4 matrix for WebGL/Three.js integration.
- `targetDimensions`: `[width, height]` of the source target.
### 5. Advanced Integration (Three.js / A-Frame)
We still provide wrappers for 3D engines if you need to render complex 3D models instead of DOM overlays.
#### Three.js Adapter
```javascript
import { TaarThree } from '@srsergio/taptapp-ar';
// ... (standard Three.js integration)
```
---
## ποΈ Protocol V11 (Nanite Virtualized Format)
TapTapp AR uses a proprietary **Nanite-style Vision Codec** that is significantly more efficient than standard AR formats.
- **Virtualized Multi-Octave Features**: Instead of storing redundant images for each scale, V11 stores a single high-res keyframe with features stratified across 6 octaves.
- **Dynamic Scale Filtering**: The tracking engine estimates the target's current scale and dynamically filters the matching search space, reducing Hamming distance ops by up to 90%.
- **Non-Rigid Surface Tracking**: Replaces the standard rigid homography with a dynamic **Delaunay Mesh**. This allows the tracker to follow the curvature of posters on cylinders, t-shirts, or slightly bent magazines.
- **Mass-Spring Relaxation**: The tracking mesh is optimized using physical relaxation, minimizing L2 distance between predicted and tracked points while maintaining topological rigidity.
- **Fourier Positional Encoding**: Maps 2D coordinates into a 16-dimensional frequency space. This creates a "Neural Consistency Check" that filters out noise and motion blur by checking for harmonic spatial agreement.
- **4-bit Packed Tracking Data**: Image data used for optical flow is compressed to 4-bit depth.
- **64-bit LSH Fingerprinting**: Feature descriptors are compressed to just 8 bytes using LSH.
- **Binary Matching Engine**: Uses hardware-accelerated population count (`popcount`) and `XOR` for near-instant point matching.
- **Zero-Copy Restoration**: Binary buffers are mapped directly to TypedArrays (Uint32 for descriptors, Float32 for tracking coordinates, Int8 for Fourier signatures).
---
## π Visual Search & Embeddings (NEW!) π
[](https://www.npmjs.com/package/@srsergio/taptapp-ar)
[](https://www.npmjs.com/package/@srsergio/taptapp-ar)
[](./LICENSE)
[](https://bundlephobia.com/package/@srsergio/taptapp-ar)
[](https://www.typescriptlang.org/)
[](https://www.npmjs.com/package/@srsergio/taptapp-ar)
### π Ultra-Fast AR Tracking β’ 100% Pure JavaScript β’ No TensorFlow Required
