Skip to content

AT-SPI: a combo box's value, and expanded/collapsed states, don't reach screen readers #824

Open

Description

@Maddimax

🤖 Maddimax (AI)

Versions: accesskit 0.25.1, accesskit_atspi_common 0.21.0 (the adapter accesskit_unix 0.24 and accesskit_winit 0.34 use).

Two things a tree provides never reach AT-SPI clients such as Orca. macOS and UI Automation both expose them already.

1. A combo box's value

A collapsed combo box gets its current text with Node::set_value. On AT-SPI the value has nowhere to go:

  • the object's name is its label, and only Role::Label takes its name from the value (label_comes_from_value);
  • the Value interface is numeric only (current_value);
  • the Text interface is only for nodes with text runs.

So Orca reads "Fruit, combo box" where VoiceOver and Narrator say "Fruit, combo box, Apple". UI Automation reads the value through the Value pattern, and macOS through accessibilityValue.

Orca reads a combo box's value from its selected child, or from its Text interface. Either of these would fix it:

  • give a combo box with a value the Text interface, holding that value;
  • or expose its selection: an object for the value, as GTK structures its combo boxes.

As a workaround we now give collapsed combo boxes a selected MenuListOption child on Linux only. We'd rather drop it.

2. Expandable, expanded and collapsed

Node::set_expanded maps to no AT-SPI state, so Orca can't tell whether a combo box's popup is open, or whether a tree item is expanded. #665 adds Expandable with Expanded or Collapsed, and with it the state-change events through notify_state_changes. This issue is mostly here to say it is wanted, and to link the combo box case above.

Related: Action::Expand and Action::Collapse aren't offered through the Action interface either. Only click is listed, below.

Reproduction

Cargo.toml:

[dependencies]
accesskit = "0.25"
accesskit_atspi_common = "0.21"

src/main.rs:

use accesskit::{
    Action, ActionHandler, ActionRequest, Node, NodeId, Rect, Role, TreeId, TreeInfo, TreeUpdate,
};
use accesskit_atspi_common::{
    Adapter, AdapterCallback, AppContext, Event, FullNodeId, InterfaceSet, PlatformNode, State,
    WindowBounds,
};

struct NoCallback;
impl AdapterCallback for NoCallback {
    fn register_interfaces(&self, _: &Adapter, _: FullNodeId, _: InterfaceSet) {}
    fn unregister_interfaces(&self, _: &Adapter, _: FullNodeId, _: InterfaceSet) {}
    fn emit_event(&self, _: &Adapter, _: Event) {}
}

struct NoActions;
impl ActionHandler for NoActions {
    fn do_action(&mut self, _: ActionRequest) {}
}

const WINDOW: NodeId = NodeId(0);
const COMBO_BOX: NodeId = NodeId(1);
const TREE_ITEM: NodeId = NodeId(2);

fn main() {
    let mut window = Node::new(Role::Window);
    window.set_children(vec![COMBO_BOX, TREE_ITEM]);

    // A collapsed combo box showing "Apple".
    let mut combo_box = Node::new(Role::ComboBox);
    combo_box.set_bounds(Rect::new(0.0, 0.0, 100.0, 30.0));
    combo_box.set_label("Fruit");
    combo_box.set_value("Apple");
    combo_box.set_expanded(false);
    combo_box.add_action(Action::Click);
    combo_box.add_action(Action::Expand);

    // An expanded tree item.
    let mut tree_item = Node::new(Role::TreeItem);
    tree_item.set_bounds(Rect::new(0.0, 40.0, 100.0, 70.0));
    tree_item.set_label("Documents");
    tree_item.set_expanded(true);
    tree_item.add_action(Action::Collapse);

    let update = TreeUpdate {
        nodes: vec![(WINDOW, window), (COMBO_BOX, combo_box), (TREE_ITEM, tree_item)],
        tree: Some(TreeInfo::new(WINDOW)),
        tree_id: TreeId::ROOT,
        focus: COMBO_BOX,
    };
    let adapter = Adapter::new(
        &AppContext::new(Some("repro".into())),
        NoCallback,
        update,
        true,
        WindowBounds::default(),
        NoActions,
    );
    let root = adapter.platform_node(adapter.root_id());
    let child = |index| root.relative(root.child_at_index(index).unwrap().unwrap());
    for node in [child(0), child(1)] {
        report(&node);
    }
}

fn report(node: &PlatformNode) {
    let state = node.state();
    println!("{:?} {:?}", node.role().unwrap(), node.name().unwrap());
    for flag in [State::Expandable, State::Expanded, State::Collapsed] {
        println!("  {flag:?}: {}", state.contains(flag));
    }
    println!("  interfaces: {:?}", node.interfaces().unwrap());
    println!("  children: {}", node.child_count().unwrap());
    let actions: Vec<String> = (0..node.n_actions().unwrap_or(0))
        .map(|i| node.action_name(i).unwrap_or_default())
        .collect();
    println!("  actions: {actions:?}");
}

Output:

ComboBox "Fruit"
  Expandable: false
  Expanded: false
  Collapsed: false
  interfaces: InterfaceSet(BitFlags<Interface>(0b100000000100011, Accessible | Action | Component | Selection))
  children: 0
  actions: ["click"]
TreeItem "Documents"
  Expandable: false
  Expanded: false
  Collapsed: false
  interfaces: InterfaceSet(BitFlags<Interface>(0b100001, Accessible | Component))
  children: 0
  actions: []

"Apple" can't be read through any interface. Neither node has Expandable, Expanded or Collapsed. Neither offers expand or collapse as an action.

Expected:

  • the combo box's value can be read: as Text, or as a selected child;
  • the combo box is Expandable and Collapsed;
  • the tree item is Expandable and Expanded;
  • the expand and collapse actions are offered where the node supports them.

Activity

  1. hr-mes commented on Nov 5, 2026

    @hr-mes

    We hit the same gap with a menu button and a text field on the Unix adapter, and would like to propose a small change to accesskit_atspi_common that covers the expanded part of this issue and adds has_popup, which is unmapped too.

    What is missing

    NodeWrapper::state (adapters/atspi-common/src/node.rs, lines 301-389 at 52d3ee4) maps neither Node::is_expanded nor Node::has_popup to an AT-SPI state. So:

    • a button with set_has_popup(HasPopup::Menu) and set_expanded(false) reaches AT-SPI with neither STATE_EXPANDABLE nor STATE_HAS_POPUP. Orca says "Files, button" where it says "Files, collapsed button ... opens popup" for the GTK 4 menu button it replaces;
    • a text input with set_has_popup(HasPopup::Menu) for its edit menu reaches AT-SPI without STATE_HAS_POPUP. GTK 4.24's entry sets it, and Orca 51 reads it as "opens popup".

    Nothing in 0.21.1 or on main changes this.

    Proposed change

    This follows Core-AAM 1.2: aria-expanded maps to STATE_EXPANDABLE, with STATE_EXPANDED when it is true, and aria-haspopup maps to STATE_HAS_POPUP. It is also what GTK 4 exposes for its menu buttons: expandable and has-popup, and no collapsed. Orca already reads an expandable object that is not expanded as "collapsed". Because notify_state_changes compares the old and new state sets, the state-changed events follow with no further change.

    --- a/adapters/atspi-common/src/node.rs
    +++ b/adapters/atspi-common/src/node.rs
    @@ -385,6 +385,16 @@ impl NodeWrapper<'_> {
                 atspi_state.insert(State::Focused);
             }
     
    +        if let Some(expanded) = state.data().is_expanded() {
    +            atspi_state.insert(State::Expandable);
    +            if expanded {
    +                atspi_state.insert(State::Expanded);
    +            }
    +        }
    +        if state.has_popup().is_some() {
    +            atspi_state.insert(State::HasPopup);
    +        }
    +
             atspi_state
         }

    #665 adds STATE_COLLAPSED as well. Core-AAM does not list it for aria-expanded="false", and Orca does not need it, so we left it out. We are happy to follow whichever you prefer. Core-AAM also lists the object attribute haspopup:<value> for the non-boolean aria-haspopup values. It could go in attributes() in the same change or a later one.

    Two related gaps are out of scope here: the combo box value in part 1 of this issue, and the expand and collapse actions.

    Evidence

    A minimal program against accesskit 0.25.1 and accesskit_atspi_common 0.21.0, along the lines of the reproduction above, builds a window with:

    • a Role::Button labelled "Files", with set_has_popup(HasPopup::Menu) and set_expanded(false);
    • a Role::TextInput labelled "Name", with set_has_popup(HasPopup::Menu).

    It then reads PlatformNode::state() for both: Expandable, Expanded, Collapsed and HasPopup are all false on both nodes.

    With the change above applied, both nodes have HasPopup, the button is Expandable, and it is also Expanded after set_expanded(true).

    We carry this change as a local patch of 0.21.0 until a release maps the states, and we will drop the patch then. A test in our toolkit reads the states through Adapter/PlatformNode. It fails without the change and passes with it.

    If you would like a pull request, with a test in adapters/atspi-common next to the existing ones, we can prepare one, as CONTRIBUTING.md describes for repository members.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions