Agent Integration

Use Pindrop with AI agents and browser automation tools

Pindrop is designed to work as a native tool for AI agents doing UI review. Instead of writing findings to a text file or Slack message, an agent pins feedback directly on the page where the issue lives.

How it works

The typical flow:

Agent calls getCommentableElements() to get a structured inventory of the page
Agent (optionally) takes a screenshot and sends both to an LLM
LLM returns structured feedback — selector, comment text
Agent calls addComment() for each item

javascript

const elements = pindrop.getCommentableElements()
 
const feedback = await llm.call({
  prompt: 'Review this UI. Return JSON: [{ selector, text }]',
  context: elements,
  screenshot: await page.screenshot(),
})
 
for (const item of feedback) {
  pindrop.addComment({
    selector: item.selector,
    text: item.text,
    author: 'Agent',
    meta: { source: 'agent', model: 'claude-sonnet-4-6' },
  })
}

Function calling tool definitions

Drop these into your agent's tool list:

json

[
  {
    "name": "get_commentable_elements",
    "description": "Returns semantic, visible elements on the page with selectors, labels, and positions. Call this first to understand the page layout.",
    "parameters": { "type": "object", "properties": {}, "required": [] }
  },
  {
    "name": "leave_comment",
    "description": "Pin a review comment on a specific element. Use selector from get_commentable_elements, or viewport coordinates (0–1) when targeting by screenshot.",
    "parameters": {
      "type": "object",
      "properties": {
        "selector": { "type": "string", "description": "CSS selector. Omit if using x/y." },
        "x": { "type": "number", "description": "Viewport X fraction (0–1). Use with y." },
        "y": { "type": "number", "description": "Viewport Y fraction (0–1). Use with x." },
        "text": { "type": "string", "description": "The comment text." },
        "author": { "type": "string", "description": "Agent name shown on the pin." }
      },
      "required": ["text"]
    }
  },
  {
    "name": "resolve_comment",
    "description": "Mark a comment as resolved after the issue has been addressed.",
    "parameters": {
      "type": "object",
      "properties": {
        "commentId": { "type": "string" }
      },
      "required": ["commentId"]
    }
  }
]

Targeting by coordinates

When an agent reasons from a screenshot, it may not have reliable CSS selectors. Pass viewport fractions instead:

javascript

pindrop.addComment({
  x: 0.7,
  y: 0.85,
  text: 'This chart looks misaligned',
  author: 'Agent',
})

Pindrop hits the element at that point and anchors to it. If nothing is found (e.g. the coordinates land on a gap between elements), it falls back to a viewport-only pin that remains visible at those coordinates.

Agent metadata

Mark comments as agent-generated using the meta field. This lets you filter, style, or route them differently:

javascript

pindrop.addComment({
  selector: '#pricing-table',
  text: 'CTA button contrast is 2.8:1 — below WCAG AA threshold of 4.5:1',
  author: 'Accessibility Bot',
  meta: { source: 'agent', model: 'gpt-4o' },
})

llms.txt

A machine-readable summary of the full API is available at /llms.txt. It includes all method signatures, type definitions, and the function calling tool definitions above — intended for agents that discover APIs dynamically.

Last updated March 29, 2026