Advanced

Screen Observer

Enable your AI assistant to understand what users are viewing in your application and provide contextual responses based on the current screen state.

What is Screen Observer?

Screen Observer allows your AI agent to see and understand the context of what users are viewing in your application. Instead of asking "What product am I looking at?", users can simply say "How much is this?" and the agent will know exactly what they're referring to.

Perfect for:

• E-commerce product pages
• Event ticketing and booking systems
• SaaS dashboards and admin panels
• Any app where context matters

How It Works

Screen Observer uses browser postMessage API to send screen state from your application to the Meebly iframe:

1. Your app sends screen state via postMessage
2. Meebly iframe receives and validates the state
3. Screen state is included in chat API requests
4. AI agent uses screen context in responses

Quick Start

Step 1: Add This Code to Your App

Copy-paste this helper function anywhere in your JavaScript:

javascript

// Add this helper function to your app
function updateMeeblyScreen(screenState) {
  const iframe = document.getElementById('meebly-agent');
  if (!iframe?.contentWindow) return;

  iframe.contentWindow.postMessage({
    type: 'MEEBLY_SCREEN_STATE',
    screenState: {
      ...screenState,
      timestamp: new Date().toISOString()
    }
  }, '*');
}

Step 2: Send Screen Updates

Call the function when users navigate or view content:

javascript

// E-commerce product page example
updateMeeblyScreen({
  route: '/products/running-shoes-123',
  routeName: 'Product Page',
  entities: {
    product: {
      id: 'running-shoes-123',
      name: 'Trail Running Shoes',
      price: 89.99,
      inStock: true
    },
    cart: {
      itemCount: 2,
      total: 150.00
    }
  }
});

Step 3: Test Contextual Responses

Ask your agent contextual questions:

User: "How much is this?"

Agent: "The Trail Running Shoes are $89.99 and are currently in stock."

User: "Can I add it to my cart?"

Agent: "Yes! I can add the Trail Running Shoes to your cart. You currently have 2 items totaling $150.00. Would you like me to proceed?"

API Reference

Screen State Object

The screenState object is included in your chat API requests:

json

{
  "agentId": "agent_abc123",
  "environmentId": "env_xyz789",
  "messages": [{"content": "How much is this?"}],
  "screenState": {
    "route": "/products/123",
    "routeName": "Product Page",
    "entities": {
      "product": {
        "id": "123",
        "name": "Running Shoes",
        "price": 89.99
      }
    },
    "timestamp": "2025-12-11T12:00:00Z"
  }
}

Screen State Fields

Field	Type	Required	Description
`route`	string	Yes	Current page route or URL path
`routeName`	string	No	Human-readable page name
`entities`	object	No	Key data visible on screen (products, users, cart, etc.)
`observedElements`	array	No	DOM elements with data-meebly attributes for precise context
`viewportSize`	object	No	Screen dimensions (width, height) for responsive context
`timestamp`	string	Yes	ISO 8601 timestamp (auto-added)

Security Note: Do not include sensitive information (passwords, credit cards, PII) in screen state. Only send data that is already visible to the user.

Complete Screen State Example

Here's a comprehensive example showing all available fields:

json

{
  "screenState": {
    "route": "/products/running-shoes-123",
    "routeName": "Product Detail Page",
    "entities": {
      "product": {
        "id": "running-shoes-123",
        "name": "Trail Running Shoes",
        "price": 89.99,
        "inStock": true,
        "category": "Athletic Footwear"
      },
      "user": {
        "id": "user_456",
        "name": "John Doe",
        "membershipLevel": "premium"
      },
      "cart": {
        "itemCount": 2,
        "total": 150.00
      }
    },
    "observedElements": [
      {
        "selector": "[data-meebly='add-to-cart']",
        "text": "Add to Cart",
        "attributes": {
          "data-product-id": "running-shoes-123",
          "data-action": "add-to-cart"
        }
      },
      {
        "selector": "[data-meebly='buy-now']",
        "text": "Buy Now",
        "attributes": {
          "data-product-id": "running-shoes-123",
          "data-action": "quick-checkout"
        }
      }
    ],
    "viewportSize": {
      "width": 1920,
      "height": 1080
    },
    "timestamp": "2025-12-21T12:00:00Z"
  }
}

Using observedElements

The observedElements field captures specific DOM elements marked with data-meebly attributes, giving agents precise context about interactive elements on the page:

html

<!-- Mark key elements with data-meebly attribute -->
<button
  data-meebly="add-to-cart"
  data-product-id="running-shoes-123"
  data-action="add-to-cart"
>
  Add to Cart
</button>

<button
  data-meebly="buy-now"
  data-product-id="running-shoes-123"
  data-action="quick-checkout"
>
  Buy Now
</button>

Then capture these elements in your screenState:

javascript

// Collect observed elements
const observedElements = Array.from(
  document.querySelectorAll('[data-meebly]')
).map(el => ({
  selector: `[data-meebly='${el.getAttribute('data-meebly')}']`,
  text: el.textContent?.trim(),
  attributes: Object.fromEntries(
    Array.from(el.attributes)
      .filter(attr => attr.name.startsWith('data-'))
      .map(attr => [attr.name, attr.value])
  )
}));

updateMeeblyScreen({
  route: window.location.pathname,
  routeName: document.title,
  entities: { product, cart },
  observedElements,
  viewportSize: {
    width: window.innerWidth,
    height: window.innerHeight
  }
});

Pro Tip: Use observedElements to give agents visibility into specific actions users can take. This enables more accurate responses like "Click the 'Add to Cart' button" instead of generic instructions.

Framework Examples

React Integration

tsx

import { useEffect } from 'react';
import { useRouter } from 'next/router';

// Helper function (add once to your app)
function updateMeeblyScreen(screenState: any) {
  const iframe = document.getElementById('meebly-agent') as HTMLIFrameElement;
  if (!iframe?.contentWindow) return;

  iframe.contentWindow.postMessage({
    type: 'MEEBLY_SCREEN_STATE',
    screenState: { ...screenState, timestamp: new Date().toISOString() }
  }, '*');
}

// React hook
export function useScreenObserver(data: any) {
  const router = useRouter();

  useEffect(() => {
    updateMeeblyScreen({
      route: router.pathname,
      routeName: document.title,
      entities: data
    });
  }, [router.pathname, data]);
}

// Usage
function ProductPage({ product }) {
  useScreenObserver({ product, cart: getCart() });
  return <div>...</div>;
}

Vue Integration

javascript

// utils/meebly.js - Helper function (add once)
function updateMeeblyScreen(screenState) {
  const iframe = document.getElementById('meebly-agent');
  if (!iframe?.contentWindow) return;

  iframe.contentWindow.postMessage({
    type: 'MEEBLY_SCREEN_STATE',
    screenState: { ...screenState, timestamp: new Date().toISOString() }
  }, '*');
}

// composables/useScreenObserver.js
import { watch, onMounted } from 'vue';
import { useRoute } from 'vue-router';
import { updateMeeblyScreen } from '@/utils/meebly';

export function useScreenObserver(data) {
  const route = useRoute();

  const updateScreen = () => {
    updateMeeblyScreen({
      route: route.path,
      routeName: route.name,
      entities: data.value
    });
  };

  watch(() => route.path, updateScreen);
  watch(data, updateScreen, { deep: true });
  onMounted(updateScreen);
}

// Usage
<script setup>
const product = ref({ id: '123', price: 89.99 });
useScreenObserver(computed(() => ({ product: product.value })));
</script>

Next.js App Router

tsx

'use client';

import { usePathname } from 'next/navigation';
import { useEffect } from 'react';

// Helper function (add to utils/meebly.ts)
function updateMeeblyScreen(screenState: any) {
  const iframe = document.getElementById('meebly-agent') as HTMLIFrameElement;
  if (!iframe?.contentWindow) return;

  iframe.contentWindow.postMessage({
    type: 'MEEBLY_SCREEN_STATE',
    screenState: { ...screenState, timestamp: new Date().toISOString() }
  }, '*');
}

export function ProductPage({ product }: { product: Product }) {
  const pathname = usePathname();

  useEffect(() => {
    updateMeeblyScreen({
      route: pathname,
      routeName: 'Product Details',
      entities: { product }
    });
  }, [pathname, product]);

  return <div>...</div>;
}

For complete documentation, examples, and advanced features, check out the full Screen Observer documentation.

PreviousError Handling NextHuman Handoff

Last updated: March 2026Report an issue