Skip to content

Android: expose Node::custom_actions as AccessibilityActions (design question: id mapping) #846

Open

Description

@aryanchoudharypro

The core crate already defines Action::CustomAction, ActionData::CustomAction(i32), CustomAction { id, description } and Node::custom_actions(), but no platform adapter currently exposes them (iOS only treats supports_action(CustomAction) as an "interactive" hint). The Android adapter could be the first to implement them, so before sending a PR I'd like to agree on how it should behave.

What I'd implement

  • For each node with supports_action(Action::CustomAction), add one AccessibilityNodeInfo.AccessibilityAction per entry in custom_actions(), using description as the label. These appear in TalkBack's Actions menu.
  • In perform_action, map an incoming custom id back to ActionRequest { action: Action::CustomAction, data: Some(ActionData::CustomAction(id)) }.
  • Only accept an id that is in the node's current custom_actions(), so stale cached node info can't trigger an action that was removed. Skip actions with an empty description.

The open question is the id mapping

CustomAction::id is an arbitrary i32 chosen by the application, while Android requires custom action ids to avoid the framework's own android.R.id ids and to be above the legacy range (ids ≤ 0x200000 are treated as legacy bitmask actions). Options I considered:

  1. Fixed offset: base + id, with id restricted to a documented range (my prototype uses 0x3f000000 + [0, 0xFFFFFF]). Stateless, round-trips exactly, and a stale id fails safely because it is checked against the node's current list. Flutter uses a similar offset scheme.
  2. Fixed slot pool (as Jetpack Compose does, 32 slots): a hard cap, and it relies on androidx resource ids that a standalone adapter doesn't have.
  3. Adapter-side table: allocate Android ids from a counter and keep a (node, custom id) map. Supports any i32, but adds state that can go stale with TalkBack's cached node info.
  4. Index in custom_actions(): no restriction on app ids, but the Android id changes whenever the list is reordered, so cached info could target the wrong action.

I lean towards option 1. Is restricting ids to a documented range acceptable, or would you prefer a different approach? If 1 is fine, I'd also document the supported range on CustomAction::id.

Related questions

  • Should the core docs say that adapters may reject ids outside a platform-specific range?
  • Should other adapters (AT-SPI actions, NSAccessibilityCustomAction) follow later, so the semantics stay consistent across platforms?

Context and disclosure

I maintain a fork used by NVGT (a game engine used by blind developers), where this is needed for TalkBack users: per-item actions on list items such as "Move up" or "Remove". I already have a working prototype in the fork, tested on a physical realme RMX5032 running Android 16 (API 36) with Google TalkBack 17.1.0 and with BackTalk: the actions appear in the TalkBack actions menu, run, and per-item actions on list items work.

I use AI tooling (Claude) for this work, including researching Android's behaviour, writing the implementation and unit tests, and reviewing it. I will review, test and take responsibility for anything I submit. I wanted to say that up front, in case the project has a position on AI-assisted contributions. If it does, I'll follow it, and I'm happy to open nothing further if it isn't welcome.

I saw that AccessKit no longer accepts external pull requests (#832), so this is an issue to agree on the design first. If the plan looks reasonable, I'd be glad to send the PR as a member.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions