Repository navigation
Add xArm6 robot learning workshop (teleop, ACT training, deploy) - #5493
Draft
Nick Hehr (HipsterBrown) wants to merge 28 commits into
Draft
Nick Hehr (HipsterBrown) wants to merge 28 commits into
Nick Hehr (HipsterBrown) wants to merge 28 commits into
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
✅ Deploy Preview for viam-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Phase 5: name the act-xarm script and the container, document the script's defaults (which match the workshop, so no arguments are needed), and replace the broken cameras quoting with + separators. Add the clock-camera failure and the measured A100 training time. Phase 6: pin viam-labs:vla 0.2.1, note that its setup needs git and curl, close the ACT-loads and input-features TODOs, correct the n_action_steps note, and add a measured CPU latency for the floor. Phase 7 and overview: fill in the steps and image_size defaults, and list git and curl as prerequisites. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
The training script has nothing xArm-specific beyond its defaults, so it is now published as devrel:act-policy with the Robot policy task type. Its arm, gripper, and cameras arguments are required and its gripper defaults follow a percentage gripper, so Phase 5 passes the xArm values explicitly again, in the app table and the CLI command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
This PR adds a new sequential workshop, Teach an xArm6 to Stack Blocks, for an upcoming hackathon. It walks through end-to-end robot learning. A person teleoperates an xArm6 follower with an SO-ARM101 leader, and Viam records each demonstration. Participants train an ACT (Action Chunking with Transformers) policy with LeRobot on a cloud GPU, then deploy it on the machine. The leader and follower are different arms on purpose, to show mixed hardware on one platform.
flowchart LR T["Teleop<br/>SO-ARM101 → xArm6<br/>(Cartesian teleop)"] --> R["Record<br/>capture control sensor<br/>+ capture-control webapp"] R --> C["Curate<br/>review:keep →<br/>sequence dataset"] C --> Tr["Train<br/>custom ACT script<br/>(cloud GPU)"] Tr --> D["Deploy<br/>viam-labs:vla<br/>policy + controller"]The pages use the existing workshop format: front matter registers the workshop, and the pages use the
checkpointandworkshop-navshortcodes. This PR changes no layout or shortcode files. Every page isdraft: trueuntil the dependencies below land. The capability tags are all existing ones, so docs-health needs no change.Evidence
/tutorials/shows two workshop cards (pick-and-place, so-arm101-palletizing).After: a draft build (
hugo --buildDrafts) shows a third card. The nav chain renders overview → Phases 1–7 → wrap-up, each page shows "Phase N of 7", and no phase pages leak into/tutorials/all/. A production build excludes the draft pages.scripts/frontmatter_rules.py, andmake build-prodall pass. Local vale was 3.26.0, not the 3.12.0 that CI pins, so CI's run is the source of truth.vla-inference-service,smolvla-so101-training, andcapture-control-webappREADMEs and code, and theteleop-cartesiandoc. A final pass reviewed the whole branch for cross-page consistency and safety.Before publishing (
TODO(dep)markers)Blocking milestone two:
viam:ufactory:gripperneeds a{"set": N, "wait": false}option onDoCommand. Todaysetpolls until the jaw settles, up to 10 seconds, which stalls the 10 Hz control loop (06:266). Add optional non-blocking gripper "set" DoCommand viam-modules/viam-ufactory-xarm#145 adds it. Once it's released, add"write_args": {"wait": false}to the Phase 6 controller's gripper config and remove the "arm freezes at a gripper change" troubleshooting row.viam-labs:vla:policy. Verified onvla0.2.1 on an Intel NUC: the policy reachedready, itsspecswere correct, and oneinferreturned 100 actions in 0.22 s on the CPU.Teleop module extraction:
rejectedcounts (02:269).viam-serverversion, currently v0.119.0 (_index:103).ACT training script and companion repo (
viam-devrel/xarm-act-stacking):devrel:act-policy0.1.0, a general ACT training script, with the Robot policy task type. Its arm, gripper, and cameras arguments are required.steps50,000 andimage_size0. These are measured on an A100, at 0.081 s per step. Phase 5 passes the xArm arguments explicitly.cell.jsonmust set the teleopgripper_invert: trueandleader_tool_offset_mm, because Phase 2 tells readers they're already set.Version pins:
viam-labs:vla0.2.1, which matches the training container's LeRobot commit.viam:capture-control(03:27).Verify on hardware:
state_dimandaction_dimof 7 on an xArm checkpoint (06:135). They were 6 and 6 on the 5-joint SO-101 checkpoint, as expected.joint_limits_degs(06:192).actions_per_chunk(06:214). On a NUC's CPU, the floor is 3.Platform checks:
additional_params.docommand_input(03:94).Also:
ASSETplaceholders with photos, screenshots, and diagrams.draft: false.Merge Danger
Door: two-way
All pages are drafts, so merging publishes nothing. Reverting is a plain revert.
Blast Radius: none
The only change outside the workshop directory is one
IgnoreURLsentry in.htmltest.yml, for the companion repo that doesn't exist yet.🤖 Generated with Claude Code
https://claude.ai/code/session_012FAJrDL3amKcRmphzeVnyb