Repository navigation
feat(linux): in-process JPEG for screenshot, with region and the Swift arguments (#32 item 5) - #82
Conversation
…t arguments (#32 item 5) Debian's grim has no libjpeg, so every Linux frame was PNG (~2 MB at 1080p). For JPEG the daemon now asks grim for raw PPM and encodes it with jpeg-encoder 0.7 (pure Rust, zero deps); grim still crops (-g) and scales (-s). screenshot now takes the macOS contract: format jpeg (default) | png, quality 0-1 (default 0.7; 1-100 accepted), scale (default 0.5), region {x,y,width,height} clipped to the output layout, display 0. Invalid arguments are -32602 instead of silently defaulting. structuredContent adds points / region / image {width,height,bytes} / display (bytes and format kept), plus a text caption like Swift. points and the clip come from the virtual pointer's layout, so they share mouse's coordinate space. Verified on rpi1 (aarch64): cargo fmt --check, cargo clippy --release --all-targets -D warnings, cargo test --release (50 passed), OAB_RELEASE_VERSION=0.8.2 cargo build --release; markdownlint-cli2 0.17.2 docs/**/*.md 0 errors. Real seat on rpi2 (labwc, 1920x1080): Connect-shaped call q0.6 scale 1 -> 50,546 B JPEG in 0.07 s vs 2,065,950 B PNG in 1.03 s. Release binary +217 KB unstripped (+196 KB stripped). Refs #32
…ointer (#32 item 5) Review round 1 (opus5): - MEDIUM: seat::layout() went through with_pointer, so a screenshot could connect a zwlr_virtual_pointer_v1 (input device, 400 ms settle, pointer mutex, region failing without virtual-pointer support). layout() now returns a live pointer's layout when the slot is free (so screenshot and mouse agree) and otherwise runs query_layout(): a short Wayland connection that binds only wl_output. The wl_output event parsing is shared with Pointer::connect via apply_output_event. - LOW: quality 0-1 is a fraction floored at 0.1 (Swift's minimum), 10-100 a percentage, and 1 < q < 10 is rejected as ambiguous. Tests, schema and README updated. - LOW (hotplug): screenshot now follows the outputs unless a pointer is live; docs say so. Verified: rpi1 (aarch64) cargo fmt --check, cargo clippy --release --all-targets -D warnings, cargo test --release (50 passed, 1 ignored), OAB_RELEASE_VERSION=0.8.2 cargo build --release (stripped 2,036,696 B, unchanged); markdownlint-cli2 0.17.2 docs/**/*.md 0 errors. rpi2 labwc seat: ignored test query_layout_reads_a_live_seat passes; scratch instance screenshot: 1080p q0.6 JPEG 50,518 B vs PNG 2,065,818 B, region clip and error cases as before.
Review round 1 →
|
…ge (#32 item 5) Round-2 review: quality values strictly between 1 and 10 returned -32602. 0..=1 stays a fraction floored at 0.1 (1 = 100%, as on macOS); values above 1 through 100 are now a rounded percentage (at least 1). Schema text, crate README and tests updated. The crate README's old rpi1 verification line no longer claims the JPEG call returns PNG. Checked on rpi1 (aarch64): cargo fmt --check OK; cargo clippy --release --all-targets -D warnings OK; cargo test --release 50 passed, 1 ignored; cargo build --release OK. markdownlint-cli2 0.17.2 docs/**/*.md: 0 errors.
|
Review round 2 (opus5 LGTM, sol CHANGES_REQUESTED on Fixed
Declined (follow-up)
Checked on rpi1: |
Final review round →
|
Conflicts: - poc/reverse-attach-linux/README.md: kept this branch's mouse/screenshot layout bullet (replacing the PNG-only bullet) and main's updated wss:// and live-pod bullets. - poc/reverse-attach-linux/Cargo.lock: took main's and let cargo add jpeg-encoder 0.7.1.
Refs #32. This PR implements item 5 only and does not close the issue.
image/jpeg≤ 300 KB at 1080p q0.6 (measured 50,546 B);structuredContent.points{width,height} present.What changes
Debian's
grimis built without libjpeg, so every Linux frame was PNG (~2 MB at 1080p). For JPEG the daemon now asks grim for raw PPM (-t ppm) and encodes it in-process withjpeg-encoder0.7. That crate is pure Rust and has no dependencies. grim still does the crop (-g) and the scaling (-s), so there is no in-process resampling. I pickedjpeg-encoderoverimagebecause we only need an encoder: there is nothing to decode (PPM is parsed in ~40 lines) and nothing to resize.screenshotnow follows the SwiftScreenshotToolcontract:formatjpegsilently returned PNGquality0.6was ignoredscaleregiondisplaystructuredContentaddsdisplay,points(layout size),region(the effective crop, or the whole layout), andimage{width,height,bytes}. The old top-levelbytesandformatstay. The result also carries a text caption after the image, as on macOS.pointsand the clip come fromseat::layout(): a live virtual pointer's layout when one exists (so screenshot andmouseuse one coordinate space), otherwise a short Wayland connection that binds onlywl_output. A screenshot never creates an input device.A malformed argument (non-number
scale, unknownformat,qualityoutside 0–100, badregion) now gets -32602. Before, it fell back to the default without saying so.The Linux default
scalestays 0.5. Connect always sendsscale 1, and the brief asked only for format and quality to match macOS.Not touched:
src/attach/client.rs,src/http.rs.Cargo.tomlgains one line, placed right under[dependencies]to keep rebases with #66 easy.Cargo.lockgains one package.Measurements
a Raspberry Pi labwc seat, 1920×1080, branch binary run as a scratch instance on loopback:
display 0, scale 1, quality 0.6, format jpegscale 1, format pngscale 0.5, format png(old default)region 0,0 800×600, scale 1region 1800,1000 400×400, scale 2(clipped to 120×80)The desktop in this test is mostly wallpaper, so a text-heavy page will encode larger. The ≤ 300 KB target still has about 6× headroom.
Release binary (aarch64,
cargo build --release): 2,607,408 → 2,824,552 B unstripped (+217 KB), 1,840,088 → 2,036,696 B stripped (+196 KB). The release tarball is a plain glibccargo build --release, and its build is unchanged.Tests
New unit tests:
frame.rs: PPM parse with comments, malformed PPM rejected, encoding a known buffer (SOI/EOI and SOF0 size checked), quality monotonic, >65535 px refused, PNG IHDR read.screen.rs: Swift defaults, Connect-shaped args, quality fraction and percent mapping, invalid args → -32602 / display 1 → -32000, region clip (inside, fractional, partial, huge, outside, negative layout origin).wlroots.rs: grim argv.Ran on a Raspberry Pi (aarch64):
cargo fmt --check,cargo clippy --release --all-targets -- -D warnings,cargo test --release(50 passed), andmarkdownlint-cli2@0.17.2 "docs/**/*.md"(0 errors).Docs:
poc/reverse-attach-linux/README.md(drops the "PNG only" limitation),docs/linux-setup.md, the top-level README platform table, and thedocs/parity-checklist.mdscreenshot row.Release note:
quality: 1means 100% (fraction form, as on macOS); integers 2–100 are percentages.