Simulating widgets

Emit interactive UI widgets from your agent and let Snowglobe simulate users clicking buttons, picking options, and filling forms.

If your agent renders UI widgets — buttons, dropdowns, multi-select pickers, text inputs — Snowglobe can simulate users interacting with them. You return widget definitions from your completion() function, and Snowglobe’s simulated user picks options, fills inputs, and clicks buttons just like a real user would. On the next turn your agent receives the user’s actions as structured data.

Use this when:

• Your agent emits forms or quick actions, and you want to test them under realistic user flows
• A button click or option selection is part of the conversation (escalation buttons, “yes/no” confirmations, order-pickers, etc.)
• You want simulations that exercise both your free-text and widget paths

Widgets are part of agent simulation. If you don’t have access yet, see the agent simulation overview for how to enable it.

How it works at a high level

Each turn is a round trip:

1. Your completion() returns a text response and (optionally) a list of widgets to show
2. Snowglobe persists the widgets and shows them to the simulated user along with the text
3. The simulated user picks an option, fills an input, or clicks a button
4. On the next turn, your completion() receives request.widget_actions describing what the user did
5. Your agent reads the actions and replies — possibly with more widgets, possibly with plain text

Free-text turns and widget-action turns interleave naturally. The simulated user can ignore non-blocking widgets and send a regular message instead, or fill out a form-style widget before continuing.

Prerequisites

• Agent simulation set up end-to-end. If you haven’t completed the agent simulation tutorial, do that first
• Snowglobe Connect SDK with widget support (the types live under snowglobe.client)
• Your agent already returns CompletionFunctionOutputs from its completion() function

---

Step 1. Import the widget types

The widget types are re-exported from snowglobe.client:

from snowglobe.client import (
    CompletionRequest,
    CompletionFunctionOutputs,
    Interaction,
    Button,
    Select,
    MultiSelect,
    Input,
    WidgetChoice,
    WidgetAction,
)

Five core types you’ll use day-to-day:

Type	What it is
Interaction	A group of widgets emitted on a single turn. Has an id, a blocking flag, and a list of widgets.
Button	A clickable button. The user “clicks” it.
Select	A single-choice dropdown. The user picks one option.
MultiSelect	A multi-choice picker. The user picks zero or more options.
Input	A text input. The user types into it.

---

Step 2. Emit widgets from your agent

To show widgets, set widgets on CompletionFunctionOutputs. Each entry is an Interaction containing one or more widget elements.

The simplest case: a single button.

snowglobe_agent.py

from snowglobe.client import (
    CompletionRequest,
    CompletionFunctionOutputs,
    Interaction,
    Button,
)


def completion(request: CompletionRequest) -> CompletionFunctionOutputs:
    user_text = request.get_prompt()

    # Your agent decides it wants to offer an "escalate to human" button.
    return CompletionFunctionOutputs(
        response="I can help, or you can talk to a human agent.",
        widgets=[
            Interaction(
                id="escalate_offer",
                blocking=False,
                title="Need a human?",
                widgets=[
                    Button(
                        id="escalate",
                        label="Escalate to human",
                        description="Hand off to a live agent.",
                    ),
                ],
            ),
        ],
    )

blocking=False means the simulated user can ignore the button and keep the conversation going with free text. blocking=True (form-style) is covered below.

---

Step 3. Read what the simulated user did

When the simulated user acts on a widget, the next call to completion() carries request.widget_actions — a list of WidgetAction objects describing what happened.

snowglobe_agent.py

def completion(request: CompletionRequest) -> CompletionFunctionOutputs:
    # Did the user click anything?
    for action in request.widget_actions:
        if action.interaction_id == "escalate_offer" and action.widget_id == "escalate":
            # User clicked "Escalate to human".
            return CompletionFunctionOutputs(
                response="Connecting you to a human agent now.",
            )

    # Otherwise treat the request as free text.
    user_text = request.get_prompt()
    ...

request.widget_actions is always a list — empty when the user’s input was free text, non-empty when the user interacted with a widget. Existing code that only reads request.messages keeps working without changes.

WidgetAction shape

Each action carries:

Field	Type	Set when
interaction_id	str	Always. Matches the id on the Interaction you emitted.
widget_id	str	Always. Matches the id on the widget the user touched.
action	"click" | "select_one" | "select_many" | "set_text"	Always. Determined by the widget kind.
option_id	str	select_one only. The id of the chosen WidgetChoice.
option_ids	list[str]	select_many only. Ids of the chosen choices. May be empty.
text_value	str	set_text only. The text the user typed. May be "" to clear.

The mapping from widget kind to action kind is fixed — you don’t need to handle unexpected combinations.

---

Step 4. Pick the right widget kind

Button — click

A simple action with no payload.

Button(
    id="confirm",
    label="Confirm",
    description="Confirm the order.",  # optional
)

Select — select_one

A single-choice picker. Provide a list of WidgetChoice objects with stable ids.

Select(
    id="card_type",
    label="Card type",
    choices=[
        WidgetChoice(id="virtual", label="Virtual"),
        WidgetChoice(id="physical", label="Physical"),
    ],
    description="Pick which card type to renew.",  # optional
    selected_option_id="virtual",                  # optional pre-selection
)

The user’s action arrives as action="select_one" with option_id set to one of the choice ids.

MultiSelect — select_many

A multi-choice picker. Same shape as Select but the user can choose multiple options (or none).

MultiSelect(
    id="dietary_tags",
    label="Dietary preferences",
    choices=[
        WidgetChoice(id="vegan", label="Vegan"),
        WidgetChoice(id="gluten_free", label="Gluten-free"),
        WidgetChoice(id="halal", label="Halal"),
    ],
    selected_option_ids=[],  # optional pre-selection
)

The action arrives as action="select_many" with option_ids set to the chosen ids (possibly an empty list).

Input — set_text

A free-text field.

Input(
    id="travel_note",
    label="Travel note",
    placeholder="Optional",  # shown when empty
    value="",                # optional pre-filled value
)

The action arrives as action="set_text" with text_value set to whatever the user typed (possibly "").

---

Step 5. Use the blocking flag for form-style flows

Set Interaction.blocking=True when the widgets together form a single submittable unit — the persona shouldn’t be able to send a free-text message until they either submit the form or explicitly cancel out of it. Snowglobe enforces this: while a blocking interaction is visible, the simulated user is restricted to widget actions or ending the conversation.

This is how you get realistic form-fill behavior. The simulated user can pick options and fill inputs in any order. Snowglobe batches those staged actions and delivers them to your completion() only when a button click commits the form.

snowglobe_agent.py

from snowglobe.client import (
    CompletionFunctionOutputs,
    CompletionRequest,
    Interaction,
    Select,
    Input,
    Button,
    WidgetChoice,
)


def completion(request: CompletionRequest) -> CompletionFunctionOutputs:
    # Was this turn a form submission?
    submitted = any(
        a.interaction_id == "card_renewal" and a.widget_id == "submit"
        for a in request.widget_actions
    )

    if submitted:
        # Pull the user's choices out of the same widget_actions list.
        card_type = next(
            (a.option_id for a in request.widget_actions if a.widget_id == "card_type"),
            None,
        )
        travel_note = next(
            (a.text_value for a in request.widget_actions if a.widget_id == "travel_note"),
            "",
        )
        return CompletionFunctionOutputs(
            response=f"Submitted a {card_type} card renewal. Note: {travel_note or 'none'}.",
        )

    # Otherwise show the form.
    return CompletionFunctionOutputs(
        response="Confirm the details and submit.",
        widgets=[
            Interaction(
                id="card_renewal",
                blocking=True,
                title="Renew card",
                description="Pick a card type, add an optional travel note, then submit.",
                widgets=[
                    Select(
                        id="card_type",
                        label="Card type",
                        choices=[
                            WidgetChoice(id="virtual", label="Virtual"),
                            WidgetChoice(id="physical", label="Physical"),
                        ],
                    ),
                    Input(
                        id="travel_note",
                        label="Travel note",
                        placeholder="Optional",
                    ),
                    Button(id="submit", label="Submit"),
                ],
            ),
        ],
    )

When the simulated user submits, your completion() receives a single call with multiple widget_actions — for example a select_one on card_type, a set_text on travel_note, and a click on submit. Read them as a batch.

Use blocking=True for forms. Use blocking=False for one-off buttons that don’t gate the conversation (suggestions, “Get help”, quick replies). The simulated user may ignore a non-blocking widget and just keep talking.

---

Step 6 (optional). Rewrite the user-side text with user_content

By default, when the simulated user submits a widget, the conversation transcript shows the user’s input as the widget action chips (e.g. “selected: virtual; clicked: submit”). Sometimes a click is semantically equivalent to typing a phrase — an “Escalate to human” button is really the user saying “I want to talk to a human.”

Set CompletionFunctionOutputs.user_content on the response that handles the widget action to rewrite the user-side text on that turn. Downstream evaluators, reports, and the conversation viewer all see the canonical phrase instead of the raw widget action.

return CompletionFunctionOutputs(
    response="Connecting you to a human agent now.",
    user_content="I'd like to talk to a human agent.",
)

Leave user_content unset (the default) when the widget action doesn’t map cleanly to a single phrase — that’s almost always the right default for forms.

---

Full round-trip example

A small agent that offers a renewal form, processes the submission, and then offers a non-blocking “Anything else?” follow-up:

snowglobe_agent.py

from snowglobe.client import (
    Button,
    CompletionFunctionOutputs,
    CompletionRequest,
    Input,
    Interaction,
    Select,
    WidgetChoice,
)


def completion(request: CompletionRequest) -> CompletionFunctionOutputs:
    actions = request.widget_actions

    # Branch 1: user submitted the renewal form.
    if any(a.widget_id == "submit" and a.interaction_id == "card_renewal" for a in actions):
        card_type = next(
            (a.option_id for a in actions if a.widget_id == "card_type"),
            None,
        )
        return CompletionFunctionOutputs(
            response=f"Your {card_type} card renewal is in.",
            user_content="Submit the renewal.",
            widgets=[
                Interaction(
                    id="anything_else",
                    blocking=False,
                    widgets=[
                        Button(id="end", label="No, that's all"),
                    ],
                ),
            ],
        )

    # Branch 2: user clicked "No, that's all".
    if any(a.widget_id == "end" for a in actions):
        return CompletionFunctionOutputs(
            response="Great. Have a good one!",
            user_content="No thanks, I'm done.",
        )

    # Branch 3: free text — offer the renewal form.
    return CompletionFunctionOutputs(
        response="I can renew your card. Confirm the details below.",
        widgets=[
            Interaction(
                id="card_renewal",
                blocking=True,
                title="Renew card",
                widgets=[
                    Select(
                        id="card_type",
                        label="Card type",
                        choices=[
                            WidgetChoice(id="virtual", label="Virtual"),
                            WidgetChoice(id="physical", label="Physical"),
                        ],
                    ),
                    Input(id="travel_note", label="Travel note", placeholder="Optional"),
                    Button(id="submit", label="Submit"),
                ],
            ),
        ],
    )

Run a simulation and Snowglobe will:

• Send a free-text opener; your agent returns the form
• Pick a card type, optionally fill the input, and click Submit; your agent processes the submission and offers the follow-up button
• Either ignore the follow-up and ask another question, or click “No, that’s all” to wrap up

Both branches show up in the conversation viewer with the widget chrome rendered just like the real UI.

---

What gets stored

After each turn:

• The widgets your agent emitted are persisted on the assistant turn and rendered in the Snowglobe conversation viewer
• The simulated user’s actions are persisted on the user side of the next turn
• If you set user_content, the user-side text shows your canonical phrase; otherwise it shows action chips

You don’t need to manage any of this — emit widgets, read actions, and Snowglobe handles persistence and rendering.

---

Tips and gotchas

• Widget ids are yours. Interaction.id, widget ids, and WidgetChoice.id are all opaque strings owned by your agent. Keep them stable across turns so the simulated user’s actions reference the right targets.
• Don’t reuse ids across siblings. Two widgets in the same Interaction must have distinct ids, and two WidgetChoice entries in the same Select / MultiSelect must have distinct ids.
• Empty values are valid. A set_text action with text_value="" and a select_many with option_ids=[] are both legal — they mean “the user cleared the field.”
• Disabled choices. Set WidgetChoice(disabled=True) to render an option that’s visible but can’t be selected. The simulated user respects this.
• Pre-selection. Select.selected_option_id and MultiSelect.selected_option_ids show the widget pre-filled. The user can still change the selection.
• Re-rendering. Each turn replaces the visible widgets. Re-emit the same Interaction if you want it to stay on screen across multiple turns.

---

What’s next

Tools tutorial

The end-to-end agent simulation setup that widgets build on top of.

Concepts

Simulation modes and how Snowglobe’s mocking layer works.