How to Build Reliable AI Agents: A Production-Ready Guide

Building an AI agent that works in a demo is one thing. Building one that reliably serves thousands of users in production is another challenge entirely. This guide covers the essential practices and patterns we've learned from deploying AI agents at scale with Meebly AI.

The Reliability Challenge

AI agents are inherently probabilistic. Unlike traditional software where if (x) { doY() } produces predictable results, agents make decisions based on language models that can:

• Misinterpret instructions
• Generate inconsistent outputs
• Fail to use tools correctly
• Produce hallucinations

Yet users expect reliability comparable to traditional software. How do we bridge this gap?

1. Start with Clear, Specific System Instructions

Your system instructions are the foundation of agent reliability. Vague instructions lead to unpredictable behavior.

Bad Example

You are a helpful customer service agent.

This is too generic. The agent doesn't know what actions it can take, what data it has access to, or how to handle edge cases.

Good Example

When creating an agent in the Meebly portal, write detailed system instructions:

You are a customer service agent for TechStore, an electronics retailer.

YOUR CAPABILITIES:
- Search products by name, category, or SKU
- Check order status and tracking
- Process returns for orders within 30 days
- Update customer account information

YOUR LIMITATIONS:
- You CANNOT process refunds (escalate to human agent)
- You CANNOT access payment information
- You CANNOT cancel orders after shipment

COMMUNICATION STYLE:
- Be professional but friendly
- Use clear, concise language
- Always confirm understanding before taking actions
- Provide order numbers and tracking links when relevant

SCREEN AWARENESS:
- You can see what page the customer is viewing via Screen Observer
- Reference visible products, orders, or data when appropriate
- Use context to provide specific, relevant help

ERROR HANDLING:
- If a product is out of stock, suggest similar alternatives
- If an order is not found, verify the order number with the customer
- If unsure about a policy, say "Let me connect you with a specialist"

This gives the agent a clear operating framework, reducing ambiguity and improving consistency.

2. Leverage Screen Observer for Context

Meebly's Screen Observer feature dramatically improves reliability by giving agents visual context of what users are seeing:

// Product page
function updateMeeblyScreen(screenState) {
  const iframe = document.getElementById('meebly-agent');
  if (!iframe?.contentWindow) return;

  iframe.contentWindow.postMessage(
    {
      type: 'MEEBLY_SCREEN_STATE',
      screenState: {
        ...screenState,
        timestamp: new Date().toISOString(),
      },
    },
    '*'
  );
}

// Send context when user views a product
updateMeeblyScreen({
  route: '/products/wireless-headphones-xyz',
  routeName: 'Product Details',
  entities: {
    product: {
      id: 'wireless-headphones-xyz',
      name: 'Premium Wireless Headphones',
      price: 149.99,
      inStock: true,
      rating: 4.5,
    },
    user: {
      hasInCart: false,
      viewedBefore: true,
    },
  },
});

Now when the user asks "Are these in stock?" or "How much are these?", the agent knows exactly what they're referring to—no ambiguity, no errors from misidentification.

3. Implement Structured Output Schemas

For functional agents that perform tasks, always use output schemas to enforce structure.

Without Schema

// Agent might return anything
const result = await fetch('https://api.meebly.ai/v1/agent/agent_123/execute', {
  method: 'POST',
  headers: {
    'X-API-Key': 'YOUR_API_KEY',
    'X-Backend-Token': 'USER_JWT',
  },
  body: JSON.stringify({
    environmentId: 'production',
    userContext: 'Process return for order #12345',
  }),
});
// Result could be: "I've processed the return" (useless)
// Or: { returnId: 'RET-789', status: 'approved' } (useful)

With Schema

Configure output schema in the Meebly portal when creating a functional agent:

{
  "type": "object",
  "properties": {
    "returnId": { "type": "string" },
    "status": { "type": "string", "enum": ["approved", "rejected", "pending"] },
    "refundAmount": { "type": "number" },
    "estimatedRefundDate": { "type": "string", "format": "date" },
    "reason": { "type": "string" }
  },
  "required": ["returnId", "status"]
}

Now your application can reliably parse and act on the results:

const result = await response.json();
// Guaranteed structure:
// {
//   data: {
//     returnId: 'RET-789',
//     status: 'approved',
//     refundAmount: 49.99,
//     estimatedRefundDate: '2024-12-20',
//     reason: 'Customer requested return within policy'
//   }
// }

4. Add Human-in-the-Loop for High-Impact Actions

Never let agents autonomously perform actions that could cause significant business impact without approval.

In the Meebly portal, when configuring tools/endpoints, enable "Requires Approval" for sensitive operations:

Portal Configuration:

• Tool: refundPayment
• Requires Approval: ✓ Enabled
• Approval Timeout: 30 seconds
• Description: Issue a refund to customer

When approval is required, the agent pauses execution and your app receives an event:

// Handle SSE stream from chat API
const lines = chunk.split('\n');
for (const line of lines) {
  if (line.startsWith('data: ')) {
    const data = JSON.parse(line.substring(6));

    if (
      data.type === 'meebly-event' &&
      data.event.type === 'tool-call-approval'
    ) {
      // Show approval UI to human agent
      showApprovalDialog({
        runId: data.event.runId,
        toolName: data.event.toolName,
        parameters: data.event.parameters,
      });
    }
  }
}

// When human approves
await fetch('https://api.meebly.ai/v1/chat/approve', {
  method: 'POST',
  headers: { 'X-API-Key': 'YOUR_API_KEY' },
  body: JSON.stringify({ runId: 'run_abc123' }),
});

// Or declines
await fetch('https://api.meebly.ai/v1/chat/decline', {
  method: 'POST',
  headers: { 'X-API-Key': 'YOUR_API_KEY' },
  body: JSON.stringify({
    runId: 'run_abc123',
    reason: 'Order outside refund window',
  }),
});

5. Implement Comprehensive Error Handling

Agents will encounter errors. Plan for them in your backend APIs and system instructions.

Backend API Error Responses

Structure errors that agents can understand:

{
  "success": false,
  "error": "PRODUCT_NOT_FOUND",
  "message": "Product ID not found in system",
  "suggestedAction": "verifyProductId",
  "userMessage": "I couldn't find that product. Can you verify the product name or ID?"
}

System Instructions for Error Handling

Include error handling in agent instructions:

ERROR RESPONSES:
- If product not found: Ask user to verify product name or ID
- If API timeout: Say "I'm experiencing a delay. Let me try again."
- If insufficient inventory: Suggest similar products that are in stock
- If unauthorized action: Politely decline and explain why

Monitor Meebly Events

The Meebly Chat API emits events for errors and warnings:

if (data.type === 'meebly-event') {
  switch (data.event.type) {
    case 'tool-execution-failed':
      console.error('Tool failed:', data.event.toolName, data.event.error);
      // Alert monitoring system
      break;
    case 'rate-limit-warning':
      console.warn('Approaching rate limit');
      break;
  }
}

6. Test with Screen Observer Context

Create comprehensive test suites that include screen state:

describe('Agent with Screen Observer', () => {
  test('handles product inquiry with screen context', async () => {
    const response = await fetch('https://api.meebly.ai/v1/chat', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'X-API-Key': TEST_API_KEY,
      },
      body: JSON.stringify({
        agentId: 'support-agent',
        environmentId: 'test',
        messages: [{ content: 'How much is this?' }],
        screenState: {
          route: '/products/headphones-123',
          entities: {
            product: {
              id: 'headphones-123',
              price: 149.99,
              name: 'Wireless Headphones',
            },
          },
        },
      }),
    });

    // Agent should reference the price from screen context
    const text = await getCompleteResponse(response);
    expect(text).toContain('149.99');
  });

  test('handles ambiguous query without screen context', async () => {
    const response = await fetch('https://api.meebly.ai/v1/chat', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json',
        'X-API-Key': TEST_API_KEY,
      },
      body: JSON.stringify({
        agentId: 'support-agent',
        environmentId: 'test',
        messages: [{ content: 'How much is this?' }],
        // No screenState provided
      }),
    });

    const text = await getCompleteResponse(response);
    // Agent should ask for clarification
    expect(text).toMatch(/which product|what item/i);
  });
});

7. Configure Tools Carefully in Portal

When adding tools/endpoints in the Meebly portal:

Tool Configuration Checklist

• Clear, detailed description explaining when to use the tool
• Accurate parameter definitions with types and constraints
• Example values for complex parameters
• Error response documentation in description
• Approval setting configured for sensitive actions
• Timeout settings appropriate for API response time
• Backend authentication (JWT tokens) configured if needed

Example Tool Configuration

Tool Name: searchProducts

Description:
Search for products by keyword, category, or filters.
Use this when customers ask to find, search for, or browse products.

When to use:
- "Show me running shoes"
- "Find wireless headphones under $100"
- "What laptops do you have?"

Parameters:
- query (string, required): Search keywords
- category (string, optional): Filter by category
- maxPrice (number, optional): Maximum price filter
- inStockOnly (boolean, optional): Only show in-stock items

Returns:
Array of products with: id, name, price, inStock, imageUrl

Errors:
- INVALID_CATEGORY: Ask user to clarify category
- NO_RESULTS: Suggest broadening search or different keywords

8. Monitor Agent Performance

Use Meebly's built-in observability:

Portal Analytics

• View agent conversation metrics
• Track tool usage and success rates
• Monitor response times
• Analyze user satisfaction

Custom Monitoring

// Track important metrics
const startTime = Date.now();

const response = await fetch('https://api.meebly.ai/v1/chat', {
  /* ... */
});

let toolsUsed = 0;
let errors = 0;

// Process SSE stream
for await (const event of parseSSE(response)) {
  if (event.type === 'meebly-event') {
    if (event.event.type === 'tool-execution-started') {
      toolsUsed++;
    }
    if (event.event.type === 'tool-execution-failed') {
      errors++;
    }
  }
}

// Log metrics
analytics.track('agent_conversation', {
  agentId: 'support-agent',
  duration: Date.now() - startTime,
  toolsUsed,
  errors,
  hasScreenContext: !!screenState,
});

9. Implement Rate Limiting

Meebly handles rate limiting at the API level, but implement your own safeguards:

// Client-side rate limiting
const rateLimiter = {
  requests: [],
  maxPerMinute: 60,

  canMakeRequest() {
    const now = Date.now();
    this.requests = this.requests.filter((time) => now - time < 60000);
    return this.requests.length < this.maxPerMinute;
  },

  recordRequest() {
    this.requests.push(Date.now());
  },
};

// Before sending message
if (!rateLimiter.canMakeRequest()) {
  showError('Please wait a moment before sending another message');
  return;
}

rateLimiter.recordRequest();
await sendChatMessage(message);

10. Version Your Agents

Treat agents like any other code - version them properly in the Meebly portal:

Portal Versioning

1. Create agent: customer-support-v1
2. Test thoroughly in development environment
3. Deploy to production
4. When making changes:
   • Create new version: customer-support-v2
   • Test in staging
   • Gradual rollout (update agentId in code)

Gradual Rollout

// Feature flag for agent version
const agentId = featureFlags.useNewAgent()
  ? 'customer-support-v2'
  : 'customer-support-v1';

const response = await fetch('https://api.meebly.ai/v1/chat', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'X-API-Key': 'YOUR_API_KEY',
  },
  body: JSON.stringify({
    agentId,
    environmentId: 'production',
    messages: [{ content: userMessage }],
    screenState: currentScreenState,
  }),
});

Real-World Checklist

Before deploying to production:

• System instructions are specific and comprehensive
• Output schemas defined for functional agents
• High-impact tools require human approval
• Error handling implemented at all levels
• Screen Observer integrated for context-aware responses
• Tool descriptions are detailed and accurate in portal
• Test suite covers happy paths, edge cases, and screen context
• Rate limiting is implemented
• Agent is versioned and rollout strategy is defined
• Monitoring and alerting configured
• Documentation is complete and up-to-date
• Team is trained on portal usage and HITL workflows

Conclusion

Building reliable AI agents with Meebly AI requires treating them as critical production systems. The probabilistic nature of LLMs doesn't excuse poor engineering practices - it demands better ones.

By following these practices, you can build agents that:

• Perform consistently across diverse inputs
• Understand context through Screen Observer
• Handle errors gracefully with structured responses
• Require approval for sensitive operations
• Scale safely to support thousands of users

Remember: Reliability isn't achieved once - it's maintained continuously through monitoring, testing, and iteration.

Key advantages of Meebly AI for reliability:

• Portal-based configuration - No code deploys to update agent behavior
• Screen Observer - Eliminates ambiguity through visual context
• Built-in HITL - Human approval workflows out of the box
• Structured outputs - Enforce consistency with schemas
• Comprehensive monitoring - Track performance in real-time

---

Ready to build production-ready AI agents? Get started with Meebly AI and leverage our built-in reliability features including HITL, Screen Observer, structured outputs, and comprehensive monitoring.