Skip to content

Provide a more helpful diagnostic on "shebang lookalikes" #137249

Closed

Description

@fmease

For the token sequence #! to start a shebang which would get stripped, it must be located at the very start of the file (modulo Unicode BOM) and not be followed by (whitespace | line_comment | block_comment)* [. If that's not given, the lexer won't remove it for the parser to digest instead. Meaning if the #! doesn't start an inner attribute, the parser will emit a relatively opaque / terse error of the form expected `[`, found $ACTUAL.

The most common trigger would be leading whitespace (incl. leading newlines). Example:

 #!/usr/bin/env -S cargo +nightly -Zscript
error: expected `[`, found `/`
 --> src/lib.rs:1:4
  |
1 |  #!/usr/bin/env -S cargo +nightly -Zscript
  |    ^ expected `[`

Ideally the parser would helpfully point out that the #! doesn't start a shebang here because $reasons.

I suspect this to get hit increasingly more often with the advent of RFC3424 Cargo scripts but we're generally still in uncommon and P-low territory I'd say.


This diagnostic was originally encountered by @epage in the context of doctests (source). Example reproducer:

//! ```
//! #!/usr/bin/env -S cargo +nightly -Zscript
//! fn main() {}
//! ```

rustdoc file.rs --test currently yields (abbreviated):

---- file.rs - (line 1) stdout ----
error: expected `[`, found `/`
 --> file.rs:2:3
  |
2 | #!/usr/bin/env rustc
  |   ^ expected `[`

error: aborting due to 1 previous error

Couldn't compile the test.

NB: Whether rustdoc should interpret the #! in #[doc = "#!..."] (notice: no leading space!) or equivalently /// #!... as the start of a shebang (right now it doesn't) assuming the doc attr/comment is sandwiched in between /// ``` of course is a separate question for T-rustdoc to decide. #! will likely never start a shebang inside doctests and that's good, too.


Once rustc supports frontmatter (#136889) which will get treated very similarly, we can think about improving diagnostics for their "lookalikes", too (i.e., detecting "stray" --- (etc.) in the parser).

Activity

  1. added
    A-diagnosticsArea: Messages for errors, warnings, and lints
    A-parserArea: The lexing & parsing of Rust source code to an AST
    D-terseDiagnostics: An error or lint that doesn't give enough information about the problem at hand.
    P-lowLow priority
    T-compilerRelevant to the compiler team, which will review and decide on the PR/issue.
    on Feb 19, 2025
  2. changed the title [-]Provide a more helpful diagnostic on "shebang lookalikes" (and "frontmatter lookalikes" soon too )[/-] [+]Provide a more helpful diagnostic on "shebang lookalikes" (and soon "frontmatter lookalikes" too)[/+] on Feb 19, 2025
  3. theemathas commented on Feb 19, 2025

    @theemathas
  4. fmease commented on Feb 19, 2025

    @fmease
    MemberAuthor

    cc #134810

    I'm not sure how that's related apart from the word "exclamation mark"? #! (Pound, Bang) falls into a different lexical / syntactic category and is much simpler to robustly detect and deal with by several orders of magnitude.

  5. epage commented on Feb 19, 2025

    @epage
    Contributor

    Overall, I feel like the lookalikes are most likely to happen in rustdoc that I suspect focusing on that can offer better results if its done before the conversion to test code. Frontmatters in rustdoc is being discussed on zulip

  6. fmease commented on Feb 19, 2025

    @fmease
    MemberAuthor

    I'm not sure I completely agree but I do see your point. As long as the checks are robust enough I'd be okay with that as well. E.g., we shouldn't start rejecting the following code:

    //! ```
    //! #!//usr/bin/env -S cargo -Zscript
    //! /* Intro /*!*/ */
    //! [deny(unused_must_use)]
    //! fn main() {}
    //! ```
    

    There's no shebang here.

    Just note that handling this inside rustc would get us two for one.

  7. Pyr0de commented on Feb 22, 2025

    @Pyr0de
    Contributor

    I'd like to give try solving this issue.
    Should I replace the current error message, produced here with another error message? and what should the error message be?

    @rustbot claim

  8. fmease commented on Feb 23, 2025

    @fmease
    MemberAuthor

    I think I would keep the message and simply attach a (subdiagnostic) note because it wouldn't be super high-confidence (since it likely admits several false positives (which is okay)). Thanks for taking a shot at it :)

  9. Pyr0de commented on Feb 24, 2025

    @Pyr0de
    Contributor

    If strip_shebang returns None here, we could try to strip the leading whitespaces(and maybe comments?) and check with strip_shebang again to know if there is a shebang with leading whitespaces to then emit a warning

    but this would not give a warning for a shebang which is written after some valid syntax like

    #![allow(unused_mut)]
    #!/usr/bin/env -S cargo +nightly -Zscript

    Is it needed to check for shebang throughout the file or just at the start?

  10. epage commented on Feb 24, 2025

    @epage
    Contributor

    @Pyr0de , If this is being used to help with errors in doctests, then a shebang can appear anywhere rustdoc would be injecting it.

    @fmease I overlooked this previously:

    I'm not sure I completely agree but I do see your point. As long as the checks are robust enough I'd be okay with that as well. E.g., we shouldn't start rejecting the following code:

    //! //! #!//usr/bin/env -S cargo -Zscript //! /* Intro /*!*/ */ //! [deny(unused_must_use)] //! fn main() {} //!

    There's no shebang here.

    Just note that handling this inside rustc would get us two for one.

    rustc's strip_shebang will not strip the shebang in that case because it looks for #! followed by a [, allowing for whitespace and comments in between.

  11. added a commit that references this issue on Mar 14, 2025
    881d237
  12. added a commit that references this issue on Mar 14, 2025
    ec824e2
  13. changed the title [-]Provide a more helpful diagnostic on "shebang lookalikes" (and soon "frontmatter lookalikes" too)[/-] [+]Provide a more helpful diagnostic on "shebang lookalikes"[/+] on Oct 22, 2025
  14. epage commented on Oct 23, 2025

    @epage
    Contributor

    While the tests show that we improved shebang handling generally, it looks like this doesn't help with rustdoc.

    diff --git c/tests/rustdoc-ui/doctest/failed-doctest-shebang.rs w/tests/rustdoc-ui/doctest/failed-doctest-shebang.rs
    new file mode 100644
    index 00000000000..8a8c572d7a1
    --- /dev/null
    +++ w/tests/rustdoc-ui/doctest/failed-doctest-shebang.rs
    @@ -0,0 +1,11 @@
    +//@ compile-flags:--test
    +//@ normalize-stdout: "tests/rustdoc-ui/doctest" -> "$$DIR"
    +//@ normalize-stdout: "finished in \d+\.\d+s" -> "finished in $$TIME"
    +//@ failure-status: 101
    +
    +/// ```
    +/// #!/usr/bin/env cargo
    +///
    +/// println!("hello");
    +/// ```
    +pub struct Foo;
    diff --git c/tests/rustdoc-ui/doctest/failed-doctest-shebang.stdout w/tests/rustdoc-ui/doctest/failed-doctest-shebang.
    stdout
    new file mode 100644
    index 00000000000..abc5e8f4d58
    --- /dev/null
    +++ w/tests/rustdoc-ui/doctest/failed-doctest-shebang.stdout
    @@ -0,0 +1,33 @@
    +
    +running 1 test
    +test $DIR/failed-doctest-shebang.rs - Foo (line 6) ... FAILED
    +
    +failures:
    +
    +---- $DIR/failed-doctest-shebang.rs - Foo (line 6) stdout ----
    +error: macro expansion ignores `{` and any tokens following
    +  --> $SRC_DIR/std/src/macros.rs:LL:COL
    +   |
    +  ::: $DIR/failed-doctest-shebang.rs:9:1
    +   |
    +LL | println!("hello");
    +   | ----------------- caused by the macro expansion here
    +   |
    +   = note: the usage of `println!` is likely invalid in item context
    +
    +error[E0601]: `main` function not found in crate `rust_out`
    +  --> $DIR/failed-doctest-shebang.rs:9:19
    +   |
    +LL | println!("hello");
    +   |                   ^ consider adding a `main` function to `$DIR/failed-doctest-shebang.rs`
    +
    +error: aborting due to 2 previous errors
    +
    +For more information about this error, try `rustc --explain E0601`.
    +Couldn't compile the test.
    +
    +failures:
    +    $DIR/failed-doctest-shebang.rs - Foo (line 6)
    +
    +test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in $TIME
  15. fmease commented on Oct 23, 2025

    @fmease
    MemberAuthor

    @epage Right, that's because we unfortunately strip shebang and frontmatter in doctests now … accidentally, CC #147014.

  16. epage commented on Oct 23, 2025

    @epage
    Contributor

    Should we re-open this since the intended use case (doctests) isn't fixed or create a new issue to track this?

  17. fmease commented on Oct 23, 2025

    @fmease
    MemberAuthor

    I would create a new issue. It does depend on how we want to solve #147014: If we end up reverting to the previous behavior of not stripping these, then PR #137619 should kick in. If we don't, then this diagnostic issue remains and we should do something about it (I'm omitting some other considerations).

    So opening a new issue is a safe choice regardless (I might do so later unless you're faster).

  18. epage commented on Oct 23, 2025

    @epage
    Contributor

    Created #148053

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

A-diagnosticsArea: Messages for errors, warnings, and lintsA-parserArea: The lexing & parsing of Rust source code to an ASTD-terseDiagnostics: An error or lint that doesn't give enough information about the problem at hand.P-lowLow priorityT-compilerRelevant to the compiler team, which will review and decide on the PR/issue.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions