# UI & input

> Buttons and choice cards on the canvas are clickable and highlight automatically on hover; register hotkeys, pick a position by clicking the ground, read what the mouse is pointing at, and know who the local player has selected. Clicks, hotkeys, spell casts, full chat text and players leaving all go into the event stream.

Source: https://war3ai.com/en/docs/ui-input/

What the [canvas](https://war3ai.com/en/docs/canvas/) draws is now **clickable**. The runtime takes over the game window's input, and external programs can use:

| Capability | In one sentence | Does the game receive it? |
|---|---|---|
| **Clickable canvas items** | Buttons, choice cards, panels: a click sends `ui.click`, and hovering highlights automatically | The click that lands on the button is **not received** |
| **Hotkeys** | Register combinations such as `F5` or `ctrl+shift+Q`; a press sends `hotkey` | Can optionally be swallowed (along with the character it produces) |
| **Ground clicks** | A click in the world sends `mouse.world` with ground coordinates | Can optionally be swallowed ("click a spot to place a tower") |
| **Mouse position** | Updated every frame: screen pixels, the ground point under the cursor, the hovered canvas item | — |
| **Selection** | Who the local player has selected; any change sends `selection.changed` | — |

All of this is **local input + local drawing**: nothing goes into the command stream, so it's safe in multiplayer. But if your callbacks change the world (spawning units, changing stats), that part is still single-player only.

## Python: g.ui

```python
ui = g.ui                                                   # on first use the runtime takes over window input
ui.button("shop", "Buy a potion (50 gold)", screen=(40, 300), on_click=lambda g, ev: buy(g))
c = ui.choice("Level up! Pick a reward", [("Strength +5", "Tougher"), ("Attack speed +20%", "Hits faster"), ("Summon wolf", "One more helper")],
              pause=True, on_pick=lambda g, i: give(g, i))  # a row of cards mid-screen; pause=True pauses the game while choosing
i = c.wait(timeout=30)                                      # or block and wait (events keep pumping meanwhile, none are lost)
ui.hotkey("F5", lambda g, ev: g.say(hero, "On it!"))        # swallowed by default
ui.hotkey("ctrl+shift+Q", on_press=..., swallow=False)
ui.mouse(on_click, capture=True, buttons=("left", "right"))  # capture ground clicks: report both left and right, and swallow them
xy = ui.pick_point("Where should the tower go?")            # blocking: next left click on the ground -> (x, y); Esc or timeout -> None
ui.cursor()                                                 # {'screen': (x, y), 'world': (x, y, z) or None, 'hover': 'shop'}
ui.toast("Wave 3 incoming!", seconds=3)
ui.close()                                                  # remove your own widgets and hotkeys; window input is handed back only when no other program is using input
g.close()                                                   # or disconnect entirely (you can also write with Game(...) as g:)
```

Callbacks take `(g, ev)` and fire when you call `g.events()` — the runners for bots and [gameplay mods](https://war3ai.com/en/docs/mods/) call it every tick. Clicks without a callback go into `ui.clicks`. An exception raised in a callback is only logged; it doesn't affect other callbacks or events.

You can also use the canvas layer directly: `g.canvas.text(..., clickable=True, hover=color)`, then pick up clicks from the event stream, where `ev.key` is the key you gave when drawing. Drawing a clickable element turns input on automatically, so you don't need to touch `g.ui` first.

**Hotkey syntax**: `F1`–`F24`, `A`–`Z`, `0`–`9`, `numpad0`–`numpad9`, `space enter esc tab backspace insert delete home end pageup pagedown left up right down`, optionally prefixed with `ctrl+`, `shift+` or `alt+`.

> **Warning**
>
> Letters and digits without a modifier key clash with chat input and the game's own shortcuts. Prefer keys the game doesn't use, such as F5–F8, or key combinations.

## New events

`g.events()` now also yields these (for all fields, see [W3P protocol](https://war3ai.com/en/docs/protocol/)):

| kind | When | Convenience fields |
|---|---|---|
| `ui.click` | An interactive canvas item was clicked | `.key` canvas key, `.button` (`'left'` / `'right'`), `.mods` modifier keys |
| `ui.hover` | The mouse moved onto / off a canvas item | `.key` (`None` when moving off) |
| `hotkey` | A registered hotkey was pressed | `.key` hotkey string, `.mods` |
| `mouse.world` | With ground clicks enabled, a click landed in the world | `.x .y` ground coordinates, `.button`, `.value` (1 = swallowed) |
| `selection.changed` | The local player's selection changed | Get the units with `g.selection()` |
| `spell.cast` | A unit cast a spell (the spell's cooldown started) | `.spell` four-character code, `.b` level, `.value` cooldown seconds, `.x .y` cast point |
| `message` | A line appeared in an on-screen message frame | `.text` full text, `.frame` which frame, `.chat` (when it's chat) |
| `player.left` | A player left or was removed after being defeated | `.player` |
| `game.ended` | Left the game | — |

## Chat and screen messages

What the player types in the chat box is read directly from the `message` event's `.chat`:

```python
for ev in g.events():
    if ev.kind == "message" and ev.chat and ev.chat["text"] == "-follow":
        ...                                    # ev.chat = {'channel': 'All', 'sender': '<player name>', 'text': '-follow'}
```

`g.messages()` keeps its own separate cursor, and game hints ("You need more farms", "Can't build there") are in there too. When writing a bot, use it to find out why a command didn't work.

## From other languages

- **Gateway**: `ui.button`, `ui.choice`, `ui.hotkey`, `ui.mouse`, `ui.cursor` and the rest are available under the same names on the [gateway](https://war3ai.com/en/docs/gateway/). A remote client can't pass callback functions, so clicks and hotkeys arrive through the event push (the `ui.click` event carries `key`).
- **Writing shared memory directly**: first send the semantic command `input_enable` (W3P opcode 74), and the runtime starts taking over input. In the input block `Local\War3Input_<pid>` you write the hotkey table and mouse switches, and it writes back the mouse position, the ground point under the cursor and the hovered item. Flag bit `0x40` on a canvas item means "interactive". See [W3P protocol](https://war3ai.com/en/docs/protocol/) for the layout.

## Several programs at once

Mods, Farsight, MCP and each gateway session may all place buttons and register hotkeys in one game at the same time without interfering with each other:

- Each program registers its own hotkeys and ground-click switch, and the SDK merges everyone's into one table for the runtime. Each key is listed only once; events go to everyone, and each program picks out its own hotkeys by key;
- `ui.close()` withdraws only your own; window input is handed back only when the last program leaves;
- If a program is killed before it can clean up: the runtime checks every 2 seconds, and once all registered programs have exited, it clears the hotkeys and ground-click interception they left behind, and the buttons they drew stop intercepting clicks.

## Measured

2026-09-25, live verification on a test instance, 16/16:

- Click a button → `ui.click` + callback, and the runtime's intercept count goes up by 1 (the game didn't receive that click); clicking outside the button triggers nothing;
- F6 → `hotkey`; click the ground → `mouse.world` (swallowed);
- Spawn a Paladin and select it → `selection.changed`, matching `g.selection()`; cast Divine Shield → `spell.cast('AHds', 1, 35.0)`;
- Map text → `message`; chat → `message`, with `.chat` parsing out the sender and the text;
- Defeat the computer player → `player.left`; end the game → `game.ended`.

Real mouse clicks on buttons and hover highlighting were also checked one by one.

## Limits and caveats

- **Position comes from the real mouse**: the game reads the position from the system cursor, so hover and `cursor()` reflect the real mouse. Interception only covers button and key presses.
- **Drawn under the mouse cursor**: Warcraft draws the cursor into the frame as part of the picture every frame. The canvas and speech bubbles are both drawn just before the step where the game draws the cursor: they cover the game UI, and the cursor covers them. Only when a frame has no cursor (hidden, or during a cinematic) do they fall back to drawing in the very last step.
- **System scaling**: if you write your own tests and post clicks with window messages, coordinates posted by a process that isn't DPI-aware get scaled up by the system (×1.5 measured at 150% scaling). Have your test program declare DPI awareness first. Real clicks are unaffected.
- **Fonts need warming up the first time**, which takes about 1 second. During that time the buttons aren't drawn yet and can't be clicked.
- **No ground clicks outside a game**: on the main menu and the score screen, `mouse.world` is neither sent nor swallowed.
- If a press was swallowed and you switch to another program or drag the mouse out of the window before releasing, the state is reset too, so the next release isn't swallowed as well.
- 1.27 has no functions for creating new game UI frames (they arrived in 1.31): the buttons and cards here are drawn by the runtime, so they can look however you like, but they don't appear in the game's own menu hierarchy.