Cut Your AI Agent Costs by 95% with Smart Filtering

The Problem: When "More Data" Becomes a Liability

Last month, a customer reached out with a challenge: their AI shopping agent was spending $4.20 per request just to process Slickdeals.net. The culprit? Our API was faithfully returning all 4,509 interactive elements on the page—navigation links, footers, social media buttons, and countless other UI elements that had nothing to do with the user's task: "Find the best deal on a laptop."

This isn't just a Slickdeals problem. Modern web applications are complex:

• Amazon product pages: 2,000+ elements
• Reddit threads: 1,500+ elements
• News sites: 800+ elements

When you're feeding these into GPT-4 at $0.03 per 1K tokens, costs spiral out of control fast. A single complex page could consume 300K tokens—that's $9 just to understand one page.

Why Traditional Solutions Don't Work

Option 1: Manual Filtering (Too Slow)

You could write custom CSS selectors for each site: ".buy-button, .price, .product-title". But this breaks when sites update their HTML, requires per-site configuration, and doesn't scale to thousands of domains.

Option 2: Random Sampling (Too Risky)

Just grab the first 100 elements? Congratulations, you now have 100 navigation links and zero Buy buttons. Random sampling loses critical UI elements your agent actually needs.

Option 3: Do Nothing (Too Expensive)

Keep paying $4-9 per request and watch your compute budget evaporate. Not viable for production.

Our Solution: Smart Filtering with Importance Scoring

We built an intelligent selection algorithm that automatically identifies the most important UI elements using a 5-metric scoring system:

After scoring, we select the top N elements and re-sort them spatially (top-to-bottom) so LLMs can reason about layout: "The Buy button is at the bottom-right, below the search input."

Real-World Results

Case Study: E-commerce Bot

A customer building a price comparison agent on Slickdeals.net saw immediate impact:

Monthly savings: $4,200 (based on 1,500 requests/month). Speed improvement: 15x faster. Accuracy: Zero loss—all critical buttons and forms preserved.

Case Study: Form Automation

Another customer needed to auto-fill job application forms across hundreds of different career sites. By filtering for allowed_roles: ["textbox", "button", "checkbox"], they reduced 300+ elements per page down to 12-20 form fields:

• Token reduction: 92%
• Cost savings: $1.80 per form
• Side benefit: Faster LLM processing means 3x higher throughput

How to Use Smart Filtering

We designed the API to be backward-compatible. Existing code keeps working unchanged. To enable smart filtering, add an options field:

Example 1: Token Budget (Simple)

1POST /v1/observe
2{
3"url": "https://amazon.com/dp/...",
4"mode": "map",
5"options": {
6  "limit": 100
7}
8}

This uses importance scoring to pick the top 100 elements. Perfect for complex e-commerce sites.

Example 2: Role-Based Filtering (Surgical)

1POST /v1/observe
2{
3"url": "https://app.example.com/signup",
4"mode": "map",
5"options": {
6  "filter": {
7    "allowed_roles": ["textbox", "button", "checkbox"]
8  }
9}
10}

Only return form elements. Great for login/signup automation.

Example 3: Modal Detection

1POST /v1/observe
2{
3"url": "https://example.com",
4"mode": "map",
5"options": {
6  "filter": {
7    "min_z_index": 100
8  }
9}
10}

Detect cookie banners, popups, and modals that block interaction.

What's Next?

Smart filtering is live in production today for all customers. We're already working on Phase 2.8 features:

• Semantic grouping: Return related elements together (form + submit button)
• Context-aware filtering: Use LLMs to understand intent ("find checkout button")
• Viewport-based filtering: Only return above-the-fold elements

Try It Today

Smart filtering is available now on all plans—even our free tier. Start with 1,000 free credits per month and see how much you can save.