Reference: combinators & atoms

The tutorial chapters build Calc with a curated slice of the combinator surface. This chapter is the catalog: every combinator method and free atom, the many/ driver builder, the error taxonomy, and the feature matrix — each entry with its real signature and a tiny compiling use.

The two core traits are ParseInput (must produce a value or fail) and TryParseInput (may also decline without consuming a valid token); both are introduced in chapter 2 and chapter 3. A plain fn(&mut InputRef<…>) -> Result<O, E> is a ParseInput, and a fn(&mut InputRef<…>) -> Result<ParseAttempt<O>, E> is a TryParseInput, so most atoms are just constructors for these shapes.

Since 0.3.0 both traits carry a defaulted completeness parameter (Cmpl = Complete), mirroring InputRef, so every signature in this reference reads unchanged; the parameter exists for when a parser must run under Partial input (chapter 9). The mode legend for this catalog: the try/consume-channel families are mode-generic — the leaf atoms (expect/any/Ident/keywords/puncts), every pass-through adapter (map*, filter*, validate*, then*, ignored, padded*, recover/skip_then_retry, spanned/sliced/ located), the try-driven collections (repeated, separated*, fold/rfold, collect), and the delimited shapes. Two families stay Complete-only this release, each pinned with its reason recorded on the impl: the decision-window class (*_while, peek_*, dispatch_*, pratt — a non-final frontier can silently truncate their peeked decision window, which would read as “construct ended”) and the CST node family (partial event semantics is a separately-deferred design). Reaching for a pinned combinator from partial code fails at the drive site — an E0277-family “not implemented for … Partial” wall (in method-call position it surfaces as E0599) — never a silent wrong parse.

How to read this reference

• Signatures are shown trimmed (the always-present Self: Sized, L: Lexer, Ctx: ParseContext bounds are elided) in text blocks; the compiling ```rust blocks below each family show minimal uses.
• Every example shares one hidden scaffold: a minimal hand-written Lexer — CharLexer — over single-character tokens (Digit, Ident, and the punctuation , ; + * ( ) [ ]), plus an Error that absorbs the whole taxonomy through From. Chapter 1 shows the real logos-based lexer.
• The examples fix a concrete context — FatalContext, whose Fatal emitter implements every capability trait — so the emitter where-clauses you see in the tutorials collapse to nothing here. To write a parser reusable across emitters, keep it generic over Ctx: ParseContext and name the capabilities it needs (see chapter 3); the reference stays concrete for brevity.

The Lang convention

Every trait, type, and function carries a language marker Lang: ?Sized = (). An atom has one spelling, generic over Lang, and reads the marker off the input it is handed — expect, try_expect, fail, Comma::try_parse, Keyword::parse_exact. An unbranded grammar writes the same call as a branded one, and neither needs a turbofish to say which Lang it meant.

The _of suffix survives on exactly one shape: a constructor with no input to read the marker from, where Lang has to be named or left to its () default at the call site — Any::of, ParserContext::of, Fatal::of, and the error constructors (UnexpectedEot::eot_of, Unclosed::paren_of, …). Repetition-count knobs (at_least, …) and the separated_by_* family never had a twin — they inherit Lang from the parser they wrap. The Parser constructors have no _of twin either: each either names Lang in its return type (Parser::new, with_parser) or reads it off the parsing function it is handed (with_parser_and_context, apply); with_context names no language at all.

Atoms — parsers from nothing

Any::of() -> Any<L, Ctx, Lang>                       // also ::spanned/::sliced/::located
expect(check) -> Expect<Classifier, Ctx>             // check: FnMut(&Token) -> Result<(), Expected<Kind>>
Empty::new() -> Empty                                 // Todo::<O>::new() -> Todo<O>
fail(f) -> Fail<F, L, O, Ctx>                         // f: FnMut() -> Error

Any also has ::spanned(), ::sliced(), and ::located() constructors that attach position/text to the token. expect is preferred over Any + filter because it produces an expected …, found … diagnostic from the Expected the classifier returns.

When the token’s payload is what you want rather than the token, the family has an InputRef form that is one named operation instead of two: try_expect_take classifies the head by reference — so the classification runs exactly once — commits it, and then moves it by value into a projection that lifts the payload out with no clone. Ok(None) follows try_expect: a definite absence, with the head left at the cache front. Where a decline commits the caller to a different parse, reach for try_expect_take_or_stop, which raises a terminal scanner stop as an error instead of folding it into absence. These are InputRef methods, not combinators — there is nothing to compose, which is why they are here beside the atoms rather than in the table.

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, Parser, ParseInput as _, parser::{Any, Empty, expect, fail}, utils::Expected};

// `Any::of()` — any one token.
fn first<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Tok, Error> {
    Any::of().parse_input(inp)
}
assert_eq!(Parser::with_parser(first).parse_str("+").unwrap(), Tok::Plus);

// `expect(check)` — a specific token, with a typed error otherwise.
fn a_plus<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Tok, Error> {
    expect(|t: &Tok| if matches!(t, Tok::Plus) { Ok(()) } else { Err(Expected::one(Kind::Plus)) })
        .parse_input(inp)
}
assert_eq!(Parser::with_parser(a_plus).parse_str("+").unwrap(), Tok::Plus);
assert!(Parser::with_parser(a_plus).parse_str("*").is_err());

// `Empty::new()` — succeed, consuming nothing; `fail(f)` — always fail.
fn nothing<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<(), Error> {
    Empty::new().parse_input(inp)
}
fn boom<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<(), Error> {
    fail(|| Error).parse_input(inp)
}
assert!(Parser::with_parser(nothing).parse_str("").is_ok());
assert!(Parser::with_parser(boom).parse_str("+").is_err());

// `try_expect_take` — classify by reference, then take the payload by value. The
// projection runs only after the head was accepted and committed, so its `Err` is a
// real error, never a decline.
fn digit_value<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    inp.try_expect_take(
        |t| matches!(t.data, Tok::Digit(_)),
        |sp| match sp.into_components() {
            (_span, Tok::Digit(n)) => Ok(n),
            _ => Err(Error),
        },
    )?
    .ok_or(Error)
}
assert_eq!(Parser::with_parser(digit_value).parse_str("7").unwrap(), 7);
assert!(Parser::with_parser(digit_value).parse_str("+").is_err());

Transforming output

All are methods on ParseInput. Each has a _with twin that additionally receives a ParseState (span/slice access) — map_with, filter_with, filter_map_with, validate_with.

map<U, F>(self, f: F) -> Map<…>                       // F: FnMut(O) -> U
filter<F>(self, f: F) -> Filter<…>                    // F: FnMut(&O) -> Result<(), Error>
filter_map<U, F>(self, f: F) -> FilterMap<…>          // F: FnMut(O) -> Result<U, Error>
validate<F>(self, f: F) -> Validate<…>                // F: FnMut(&O) -> Result<(), Error>
unwrap(self) -> Unwrapped<…>                          // where Self: ParseInput<Option<O>>

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, Parser, ParseInput as _, ParseInputUnwrapExt as _, parser::{expect, opt}, utils::Expected};
use tokora::try_parse_input::ParseAttempt;

fn a_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Tok, Error> {
    expect(|t: &Tok| if matches!(t, Tok::Digit(_)) { Ok(()) } else { Err(Expected::one(Kind::Digit)) })
        .parse_input(inp)
}
fn try_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<ParseAttempt<u32>, Error> {
    Ok(match inp.try_expect(|t| matches!(t.data(), Tok::Digit(_)))? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => ParseAttempt::Accept(n), _ => unreachable!() },
        None => ParseAttempt::Decline,
    })
}

// filter_map: token → value, or fail
fn value<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    a_digit.filter_map(|t| match t { Tok::Digit(n) => Ok(n), _ => Err(Error) }).parse_input(inp)
}
assert_eq!(Parser::with_parser(value).parse_str("7").unwrap(), 7);

// map + validate: value, then assert it is even
fn even<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    a_digit
        .map(|t| match t { Tok::Digit(n) => n, _ => 0 })
        .validate(|n: &u32| if n % 2 == 0 { Ok(()) } else { Err(Error) })
        .parse_input(inp)
}
assert_eq!(Parser::with_parser(even).parse_str("4").unwrap(), 4);
assert!(Parser::with_parser(even).parse_str("5").is_err());

// unwrap: an `opt` Option output, unwrapped
fn required<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    opt(try_digit).unwrap().parse_input(inp)
}
assert_eq!(Parser::with_parser(required).parse_str("9").unwrap(), 9);

Shaping — spans, slices, ignoring

These methods on ParseInput re-shape the output without changing what is consumed. spanned/sliced/located are taught in chapter 3.

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, Parser, ParseInput as _, parser::Any, span::Spanned};

// `.spanned()` wraps the output with the span it covers.
fn spanned<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Spanned<Tok, SimpleSpan>, Error> {
    Any::of().spanned().parse_input(inp)
}
let sp = Parser::with_parser(spanned).parse_str("+").unwrap();
assert_eq!(sp.data, Tok::Plus);
assert_eq!((sp.span.start(), sp.span.end()), (0, 1));

// `.ignored()` keeps the consumption, drops the value.
fn ignored<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<(), Error> {
    Any::of().ignored().parse_input(inp)
}
assert!(Parser::with_parser(ignored).parse_str("+").is_ok());

Sequencing

All methods on ParseInput. A delimited shape is just sequencing with the brackets ignored — open.ignore_then(body).then_ignore(close) — packaged ready-made by delimited and parens/braces/brackets/angles (see Delimited shapes).

then<T, U>(self, second: T) -> Then<…>               // second: ParseInput<U>
then_ignore<G, U>(self, second: G) -> ThenIgnore<…>
ignore_then<G, U>(self, second: G) -> IgnoreThen<…>
then_value<F, U>(self, value: F) -> ThenValue<…>     // value: FnMut() -> U
and_then<T, U>(self, f: T) -> AndThen<…>             // f: FnMut(O) -> Result<U, Error>

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, Parser, ParseInput as _};

fn digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? { Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) }, None => Err(Error) }
}
fn plus<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<(), Error> {
    match inp.next()? { Some(sp) if matches!(sp.data(), Tok::Plus) => Ok(()), _ => Err(Error) }
}

fn pair<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<(u32, u32), Error> {
    digit.then(digit).parse_input(inp)                    // both outputs
}
fn lhs<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    digit.then_ignore(plus).parse_input(inp)              // keep the digit, drop the `+`
}
fn rhs<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    plus.ignore_then(digit).parse_input(inp)              // drop the `+`, keep the digit
}
fn tagged<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<&'static str, Error> {
    plus.then_value(|| "op").parse_input(inp)             // consume `+`, yield a fixed value
}
fn nonzero<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    digit.and_then(|n| if n > 0 { Ok(n) } else { Err(Error) }).parse_input(inp)
}

assert_eq!(Parser::with_parser(pair).parse_str("12").unwrap(), (1, 2));
assert_eq!(Parser::with_parser(lhs).parse_str("1+").unwrap(), 1);
assert_eq!(Parser::with_parser(rhs).parse_str("+1").unwrap(), 1);
assert_eq!(Parser::with_parser(tagged).parse_str("+").unwrap(), "op");
assert!(Parser::with_parser(nonzero).parse_str("0").is_err());

Optional & choice

opt(p) adapts a declining try_-parser into one that yields Option (Some on accept, None on decline, with nothing consumed on decline). Choice is a tuple impl — a tuple of up to 32 parsers is a ParseChoice, and you drive it with one of the deterministic selectors (taught in chapter 4):

The two macros are pure syntax; their semantics live in dispatch_take and try_dispatch_take, which are public and callable directly. Both take the context bound one tier up from the rest of this section — ComposableParseContext rather than bare ParseContext — because a miss has to be expressible as an error, and both run under either Completeness.

opt and .accepted() (which turns a TryParseInput into a ParseInput<ParseAttempt<O>>/ParseInput<Option<O>>) are the bridges between the two trait worlds. On the value side, ParseAttempt::into_option is the same Accept/Decline → Some/None projection with a name — and it is the composition point for the rest of Option’s vocabulary (ok_or, filter, unwrap_or_default), which is why no aliases for those ship beside it.

select!(inp,     { Kind::A => (span, Tok::A(x)) => expr, … }) -> Result<O, Error>
try_select!(inp, { Kind::A => (span, Tok::A(x)) => expr, … }) -> Result<ParseAttempt<O>, Error>
dispatch_take(inp, table: &'static [Kind], project)           -> Result<O, Error>
try_dispatch_take(inp, table, project)                        -> Result<ParseAttempt<O>, Error>
// project: FnOnce(Spanned<Token, Span>) -> Result<O, Spanned<Token, Span>>
//   `Err(token)` is the kind-matched-but-variant-did-not arm: the token is handed BACK and
//   the runtime builds the whole-table `UnexpectedToken`, so no arm writes `unreachable!()`.
// The kind expressions must be const-promotable (a unit-variant path or a `const`) — the
// expansion hands a `&'static [Kind]` to the runtime, and anything else is `E0716` at the
// invocation.

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, ParseChoice as _, ParseInput as _, Parser, parser::opt};
use tokora::try_parse_input::ParseAttempt;

fn try_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<ParseAttempt<u32>, Error> {
    Ok(match inp.try_expect(|t| matches!(t.data(), Tok::Digit(_)))? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => ParseAttempt::Accept(n), _ => unreachable!() },
        None => ParseAttempt::Decline,
    })
}

// `opt`: Some on a digit, None (nothing consumed) otherwise.
fn maybe_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Option<u32>, Error> {
    opt(try_digit)(inp)
}
assert_eq!(Parser::with_parser(maybe_digit).parse_str("3").unwrap(), Some(3));
assert_eq!(Parser::with_parser(maybe_digit).parse_str("+").unwrap(), None);

// `dispatch_on_kind`: a static table names each branch's first token.
fn digit_branch<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? { Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) }, None => Err(Error) }
}
fn plus_branch<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    inp.next()?;
    Ok(0)
}
fn choose<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    static TABLE: [Kind; 2] = [Kind::Digit, Kind::Plus];
    (digit_branch, plus_branch).dispatch_on_kind(&TABLE).parse_input(inp)
}
assert_eq!(Parser::with_parser(choose).parse_str("8").unwrap(), 8);
assert_eq!(Parser::with_parser(choose).parse_str("+").unwrap(), 0);
assert!(Parser::with_parser(choose).parse_str(";").is_err()); // `;` is in no table slot

// `select!`: the same decision with the table written next to the patterns, and the
// classified head handed to its arm by value — `Tok::Digit(n)` binds the payload.
fn select_value<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    tokora::select!(inp, {
        Kind::Digit => (_span, Tok::Digit(n)) => n,
        Kind::Plus  => (_span, Tok::Plus)     => 0,
    })
}
assert_eq!(Parser::with_parser(select_value).parse_str("8").unwrap(), 8);
assert_eq!(Parser::with_parser(select_value).parse_str("+").unwrap(), 0);
assert!(Parser::with_parser(select_value).parse_str(";").is_err()); // committed miss

// `try_select!`: the same table, declining instead of committing — and `into_option`
// names the `Accept`/`Decline` -> `Some`/`None` projection.
fn try_select_value<'a>(
    inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>,
) -> Result<Option<u32>, Error> {
    Ok(tokora::try_select!(inp, {
        Kind::Digit => (_span, Tok::Digit(n)) => n,
        Kind::Plus  => (_span, Tok::Plus)     => 0,
    })?
    .into_option())
}
assert_eq!(Parser::with_parser(try_select_value).parse_str("8").unwrap(), Some(8));
assert_eq!(Parser::with_parser(try_select_value).parse_str(";").unwrap(), None);

Repetition & folding

The repeated driver runs a TryParseInput element until it declines; repeated_while runs a plain ParseInput element until your peek-window condition says Stop. collect accumulates the elements into any Container (a Vec, a bounded array, …). The fold family combines elements without an intermediate container.

repeated(self) -> Repeated<…>                          // element: TryParseInput
collect(self) -> Collect<…>                            // on a repetition/separation driver
fold<Init, Acc>(self, init: Init, acc: Acc) -> Fold<…> // Init: FnMut() -> O, Acc: FnMut(O, O) -> O
while_head(pred) -> WhileHead<F>                       // pred: FnMut(&Token) -> bool
while_kind(kind) -> WhileKind<K>                       // K: PartialEq<Token::Kind>

The two condition constructors are the grammar-vocabulary spelling of the cond argument that repeated_while, separated_while and the fold_while family take. Each returns a concrete Decision pinned at width 1, which is what removes the turbofish: the driver’s window parameter infers from the adapter, so the call site carries no ::<_, U1> and the hook never sees a Peeked. Note one asymmetry the type system imposes: a closure written inline for while_head needs its parameter ascribed (|t: &Tok| …), because a bare closure parameter does not infer through an impl-side Fn bound; a named function needs none, and while_kind needs neither. “Until” is the same adapter with one !.

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{
    Accumulator as _, Parse, ParseInput as _, Parser, TryParseInput as _, while_head, while_kind,
};
use tokora::try_parse_input::ParseAttempt;

fn try_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<ParseAttempt<u32>, Error> {
    Ok(match inp.try_expect(|t| matches!(t.data(), Tok::Digit(_)))? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => ParseAttempt::Accept(n), _ => unreachable!() },
        None => ParseAttempt::Decline,
    })
}

// `repeated().collect()`: zero or more digits into a `Vec`.
fn digits<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    try_digit.repeated().collect().parse_input(inp)
}
assert_eq!(Parser::with_parser(digits).parse_str("123").unwrap(), vec![1, 2, 3]);

// `fold`: sum the digits without an intermediate container.
fn sum<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    try_digit.fold(|| 0, |acc, n| acc + n).parse_input(inp)
}
assert_eq!(Parser::with_parser(sum).parse_str("123").unwrap(), 6);

// `repeated_while` + `while_head`: a *committed* element, stopped by a head predicate.
// The element cannot decline, so the loop needs a decision of its own.
fn digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) },
        None => Err(Error),
    }
}
fn digits_until_semi<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    digit
        .repeated_while(while_head(|t: &Tok| !matches!(t, Tok::Semi)))
        .collect()
        .parse_input(inp)
}
assert_eq!(Parser::with_parser(digits_until_semi).parse_str("123;").unwrap(), vec![1, 2, 3]);

// `while_kind` keys the same decision on the head's kind, and needs no ascription.
fn digit_run<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    digit.repeated_while(while_kind(Kind::Digit)).collect().parse_input(inp)
}
assert_eq!(Parser::with_parser(digit_run).parse_str("12+").unwrap(), vec![1, 2]);

Separation — comma-separated and friends

separated drives a declining element between a typed separator Punctuator; the ready-made spellings separated_by_comma and its family (separated_by_semicolon, _colon, _pipe, … 18 in all) fix the separator. Wire your token to the vocabulary with one PunctuatorToken impl (which kind is the comma) and a From<Comma<(), (), ()>> for your kind (so the punctuator can name itself). When your element cannot decline, separated_while (and the separated_by_*_while spellings) take an explicit peek condition instead.

separated<Sep>(self) -> Separated<…>                   // Sep: Punctuator; element: TryParseInput
separated_by_comma(self) -> Separated<Self, Comma, …>  // + _semicolon/_colon/_pipe/… (18)
separated_while<Sep, Cond, W>(self, cond: Cond) -> SeparatedWhile<…>  // element: ParseInput

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Accumulator as _, Parse, ParseInput as _, Parser, TryParseInput as _};
use tokora::try_parse_input::ParseAttempt;

fn try_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<ParseAttempt<u32>, Error> {
    Ok(match inp.try_expect(|t| matches!(t.data(), Tok::Digit(_)))? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => ParseAttempt::Accept(n), _ => unreachable!() },
        None => ParseAttempt::Decline,
    })
}

// `separated_by_comma`: a comma list of digits.
fn csv<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    try_digit.separated_by_comma().collect().parse_input(inp)
}
assert_eq!(Parser::with_parser(csv).parse_str("1,2,3").unwrap(), vec![1, 2, 3]);
assert!(Parser::with_parser(csv).parse_str("1,,2").is_err()); // a doubled separator is a structured failure

The many/ builder surface

Each repetition/separation driver (Repeated, Separated, SeparatedWhile) exposes a small builder to tune element counts, separator policy, and delimiters before you collect. The knobs return wrapper types (AtLeast, AllowTrailing, Bounded, …) that themselves chain, so order is flexible.

Trailing/leading violations report through dedicated emitter capabilities (UnexpectedTrailingSeparatorEmitter, …). The separator and delimiter hooks — SeparatorHandler and DelimiterHandler — are how a container observes the separators/brackets it stepped over: they are blanket-implemented as no-ops for every standard container (Vec, GenericArrayDeque, heapless, smallvec, tinyvec), so a plain collect() never has to mention them. Implement them on a custom accumulator to retain separator spans.

// on Separated (also Repeated, minus the separator knobs)
at_least(self, n: usize)        allow_trailing(self)     allow_leading(self)
at_most(self, n: usize)         require_trailing(self)   require_leading(self)
bounded(self, min, max)         delimited::<Delim>(self) -> DelimitedBy<Self, Delim>

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Accumulator as _, Parse, ParseInput as _, Parser, TryParseInput as _, punct::Bracket};
use tokora::try_parse_input::ParseAttempt;

fn try_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<ParseAttempt<u32>, Error> {
    Ok(match inp.try_expect(|t| matches!(t.data(), Tok::Digit(_)))? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => ParseAttempt::Accept(n), _ => unreachable!() },
        None => ParseAttempt::Decline,
    })
}

// bounds + a trailing separator, on a comma list
fn bounded_csv<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    try_digit.separated_by_comma().allow_trailing().at_least(1).collect().parse_input(inp)
}
assert_eq!(Parser::with_parser(bounded_csv).parse_str("1,2,").unwrap(), vec![1, 2]);

// `delimited::<Bracket>()`: a `[ … ]`-wrapped run of elements (no separators)
fn bracketed<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    try_digit.repeated().delimited::<Bracket>().collect().parse_input(inp)
}
assert_eq!(Parser::with_parser(bracketed).parse_str("[1 2 3]").unwrap(), vec![1, 2, 3]);
// An unterminated list reports the opener as `Unclosed` through the emitter — a hard
// error under this fail-fast context; a recovering emitter (`Verbose`) records the
// diagnostic and yields the elements collected so far.
assert!(Parser::with_parser(bracketed).parse_str("[1 2").is_err());

Ready-made list atoms

For the common one-liners there are free functions (alloc/std) that assemble the drivers for you and collect into a Vec — each with a fluent method twin on ParseInput that delegates to it:

The method is named list_until rather than list because the until argument is the distinguishing half, and separated1_by follows the separated_by_* family it sits beside. Both inherit their availability from the free function they delegate to: complete inputs only, and the context must be a ComposableParseContext. Streaming callers keep the element-level primitives.

One convention governs the free atoms, here and in Delimited shapes below: an atom takes its sub-parser as impl ParseInput — a closure, a fn item, or any named implementor (opt takes an impl TryParseInput attempt) — and hands back a builder-form closure that is itself a ParseInput through the blanket impl, so atoms nest into each other and into the method combinators without adapters. Predicate parameters — peek, until, and friends, which inspect a token and answer bool — are functions, not parsers, and stay plain closures.

separated1<Sep, …>(item: P, peek: Peek) -> impl FnMut(&mut InputRef) -> Result<Vec<T>, Error>
list<…>(item: P, until: Until) -> impl FnMut(&mut InputRef) -> Result<Vec<T>, Error>
   .separated1_by::<Sep, _>(self, peek: Peek) -> impl FnMut(&mut InputRef) -> Separated1Of<…>
   .list_until(self, until: Until)            -> impl FnMut(&mut InputRef) -> ListOf<…>

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, ParseInput as _, Parser, parser::{list, separated1}};

fn digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? { Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) }, None => Err(Error) }
}

// `separated1`: one-or-more comma-separated digits (optional leading comma).
fn sep_digits<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    separated1::<Comma, _, _, _, _, _, _>(digit, |t| matches!(t, Tok::Digit(_)))(inp)
}
assert_eq!(Parser::with_parser(sep_digits).parse_str(",1,2,3").unwrap(), vec![1, 2, 3]);

// `list`: zero-or-more digits until the `]` (which is left in place).
fn run<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    list(digit, |t| matches!(t, Tok::RBracket))(inp)
}
assert_eq!(Parser::with_parser(run).parse_str("123]").unwrap(), vec![1, 2, 3]);

// The method twins are the same two atoms, spelled fluently. `separated1_by` names its
// separator with a turbofish — nothing else in the call site mentions `Sep`.
fn sep_digits_fluent<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    digit.separated1_by::<Comma, _>(|t| matches!(t, Tok::Digit(_)))(inp)
}
assert_eq!(Parser::with_parser(sep_digits_fluent).parse_str(",1,2,3").unwrap(), vec![1, 2, 3]);

fn run_fluent<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Vec<u32>, Error> {
    digit.list_until(|t| matches!(t, Tok::RBracket))(inp)
}
assert_eq!(Parser::with_parser(run_fluent).parse_str("123]").unwrap(), vec![1, 2, 3]);

Delimited shapes

A committed single-region shape: commit the opener, run the inner sub-parser, commit the closer, and return a span-carrying Delimited — the open value, the close value, the inner output, and the whole-construct span. A missing closer reports the opener as Unclosed through the emitter — the same four-way close-miss law the many-builders follow: end of input with the opener still open fires Unclosed (a fail-fast emitter turns it into Err, a recovering one records it and yields the construct recovered with a synthesized closer); a wrong token where the closer belongs stays the unexpected-token (expected-close) diagnostic; a terminal scanner stop surfaces the committed form’s end-of-input error, marked terminal, once the emitter has accepted the trip’s diagnostic — a fatal emitter’s rejection of it propagates from the scan itself instead, as that emitter’s own unmarked value. This family fires only Unclosed, never the Unopened/Undelimited half of the recovery vocabulary (see Error taxonomy).

delimited::<D, …>(inner) takes the delimiter pair as its first type parameter through the TypedDelimiter capability; parens/braces/brackets/angles fix that pair to a built-in, and parens(inner) ≡ delimited::<Paren, …>(inner) for any vocabulary whose two capability declarations agree. Bring your own pair by implementing TypedDelimiter for it. This is the single-region counterpart to the many-builder’s delimited::<D>(), which instead wraps a repetition and hands its delimiter tokens to a handler.

Every shape has an attempt twin that declines — Ok(None), zero consumption — iff the opener is absent: a wrong token or end of input at entry. The moment the opener is consumed the parse is committed and every later diagnostic behaves exactly as the committed form’s — an unterminated group reports the opener as Unclosed through the emitter, never a silent decline. The attempt boundary is deliberately the opener alone, not the whole shape: opt(parens(inner)) would swallow an unclosed group into a decline, where Ident< at end of input must report Unclosed rather than silently disappear.

delimited<D, …>(inner: P) -> impl FnMut(&mut InputRef) -> Result<Delimited<Open, Close, T>, Error>
parens(inner) / braces(inner) / brackets(inner) / angles(inner) -> the same, with the pair fixed
try_delimited<D, …>(inner) / try_parens(inner) / … -> the same, wrapped in Option (None iff the opener is absent)

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, OpenBrace, CloseBrace, OpenAngle, CloseAngle, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Comma, Semi, LParen, RParen, LBracket, RBracket, LBrace, RBrace, LAngle, RAngle }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Comma, Semi, LParen, RParen, LBracket, RBracket, LBrace, RBrace, LAngle, RAngle }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Comma => Kind::Comma, Tok::Semi => Kind::Semi,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket,
    Tok::LBrace => Kind::LBrace, Tok::RBrace => Kind::RBrace,
    Tok::LAngle => Kind::LAngle, Tok::RAngle => Kind::RAngle } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
  fn open_brace() -> Option<Kind> { Some(Kind::LBrace) }
  fn close_brace() -> Option<Kind> { Some(Kind::RBrace) }
  fn open_angle() -> Option<Kind> { Some(Kind::LAngle) }
  fn close_angle() -> Option<Kind> { Some(Kind::RAngle) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
impl From<OpenBrace<(), (), ()>> for Kind { fn from(_: OpenBrace<(), (), ()>) -> Self { Kind::LBrace } }
impl From<CloseBrace<(), (), ()>> for Kind { fn from(_: CloseBrace<(), (), ()>) -> Self { Kind::RBrace } }
impl From<OpenAngle<(), (), ()>> for Kind { fn from(_: OpenAngle<(), (), ()>) -> Self { Kind::LAngle } }
impl From<CloseAngle<(), (), ()>> for Kind { fn from(_: CloseAngle<(), (), ()>) -> Self { Kind::RAngle } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      '{' => Tok::LBrace, '}' => Tok::RBrace, '<' => Tok::LAngle, '>' => Tok::RAngle,
      c => Tok::Digit(c as u32),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
fn digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? { Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) }, None => Err(Error) }
}
use tokora::{Parse, Parser, punct::Paren, parser::{braces, delimited, parens, try_parens}};

// `parens` wraps ONE region and keeps the typed delimiter values and the whole-construct span.
fn in_parens<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<(u32, SimpleSpan), Error> {
    let d = parens(digit)(inp)?;
    Ok((*d.data(), d.span()))
}
let (value, span) = Parser::with_parser(in_parens).parse_str("(1)").unwrap();
assert_eq!(value, 1);
assert_eq!(span, SimpleSpan::new(0, 3));

// `braces` fixes the pair to `{ … }`.
fn in_braces<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    braces(digit)(inp).map(|d| *d.data())
}
assert_eq!(Parser::with_parser(in_braces).parse_str("{1}").unwrap(), 1);

// `parens(inner)` ≡ `delimited::<Paren, …>(inner)`.
fn via_generic<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    delimited::<Paren, _, _, _, _, _, _>(digit)(inp).map(|d| *d.data())
}
assert_eq!(Parser::with_parser(via_generic).parse_str("(1)").unwrap(), 1);

// The attempt twin declines with zero consumption when the opener is absent…
fn attempt<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Option<u32>, Error> {
    try_parens(digit)(inp).map(|d| d.map(|d| *d.data()))
}
assert_eq!(Parser::with_parser(attempt).parse_str("1").unwrap(), None);
// …but once `(` is consumed it is committed: an unterminated group errors, it
// does not decline.
assert!(Parser::with_parser(attempt).parse_str("(1").is_err());

CST bracketing

The node family wraps everything a sub-parse commits into one syntax node over the emitter’s event channel — the lossless-CST building block (see the lossless-CST chapter, and [crate::cst]). Because the event channel is defaulted to no-ops on the diagnostic emitters, these compile and run tree-lessly over FatalContext: the wrap is inert and the output is just the inner parser’s value.

node(kind: u16, p: P) -> Node<P>              // P: ParseInput  (or TryParseInput)
node_opt(kind: u16, p: P) -> NodeOpt<P>       // P: TryParseInput -> ParseInput<Option<O>>
node_at(mark: EventMark, kind: u16, p: P) -> NodeAt<P>

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, ParseInput as _, Parser, parser::{node, node_opt}};
use tokora::try_parse_input::ParseAttempt;

const NUMBER: u16 = 1;

fn digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? { Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) }, None => Err(Error) }
}
fn try_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<ParseAttempt<u32>, Error> {
    Ok(match inp.try_expect(|t| matches!(t.data(), Tok::Digit(_)))? {
        Some(sp) => match sp.into_data() { Tok::Digit(n) => ParseAttempt::Accept(n), _ => unreachable!() },
        None => ParseAttempt::Decline,
    })
}

// tree-less over `Fatal`: the wrap is inert, the value flows through.
fn number<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    node(NUMBER, digit).parse_input(inp)
}
fn opt_number<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<Option<u32>, Error> {
    node_opt(NUMBER, try_digit).parse_input(inp)
}
assert_eq!(Parser::with_parser(number).parse_str("5").unwrap(), 5);
assert_eq!(Parser::with_parser(opt_number).parse_str("+").unwrap(), None);

Wrapping parsers

recover/inplace_recover/skip_then_retry are the error-recovery surface taught in chapter 8; labelled feeds Verbose diagnostics (see chapter 7) and is a no-op over a non-collecting emitter.

use core::{convert::Infallible, fmt};
use tokora::{
  FatalContext, InputRef, Lexer, SimpleSpan, Token,
  error::{Unclosed, UnexpectedEot, syntax::{FullContainer, MissingSyntax, TooFew, TooMany}, token::{MissingToken, SeparatedError, UnexpectedToken}},
  punct::{Comma, OpenBracket, CloseBracket, OpenParen, CloseParen, Semicolon},
  span::Span as _,
  token::PunctuatorToken,
};
#[derive(Debug, PartialEq)]
struct Error;
impl From<Infallible> for Error { fn from(e: Infallible) -> Self { match e {} } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<UnexpectedToken<'a, T, K, S, Lang>> for Error { fn from(_: UnexpectedToken<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, T, K: Clone, S, Lang: ?Sized> From<SeparatedError<'a, T, K, S, Lang>> for Error { fn from(_: SeparatedError<'a, T, K, S, Lang>) -> Self { Error } }
impl<'a, K: Clone, O, Lang: ?Sized> From<MissingToken<'a, K, O, Lang>> for Error { fn from(_: MissingToken<'a, K, O, Lang>) -> Self { Error } }
impl<O, Lang: ?Sized, Set: Clone + 'static> From<UnexpectedEot<O, Lang, Set>> for Error { fn from(_: UnexpectedEot<O, Lang, Set>) -> Self { Error } }
impl<O, Lang: ?Sized> From<MissingSyntax<O, Lang>> for Error { fn from(_: MissingSyntax<O, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<FullContainer<S, Lang>> for Error { fn from(_: FullContainer<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooFew<S, Lang>> for Error { fn from(_: TooFew<S, Lang>) -> Self { Error } }
impl<S, Lang: ?Sized> From<TooMany<S, Lang>> for Error { fn from(_: TooMany<S, Lang>) -> Self { Error } }
impl<D, S, Lang: ?Sized> From<Unclosed<D, S, Lang>> for Error { fn from(_: Unclosed<D, S, Lang>) -> Self { Error } }
impl<'a, L: Lexer<'a>, Lang: ?Sized> tokora::emitter::FromUnclosed<'a, L, Lang> for Error { fn from_unclosed<D>(_: Unclosed<D, L::Span, Lang>) -> Self { Error } }
impl tokora::error::MaybeIncomplete for Error {}
impl tokora::error::MaybeTerminal for Error {}
#[derive(Debug, Clone, PartialEq)]
enum Tok { Digit(u32), Ident(char), Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
enum Kind { Digit, Ident, Comma, Semi, Plus, Star, LParen, RParen, LBracket, RBracket }
impl fmt::Display for Kind { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { write!(f, "{self:?}") } }
impl Token<'_> for Tok {
  type Kind = Kind;
  type Error = Infallible;
  const SCAN_LOOKAHEAD: tokora::ScanLookahead = tokora::ScanLookahead::Unbounded;
  fn kind(&self) -> Kind { match self {
    Tok::Digit(_) => Kind::Digit, Tok::Ident(_) => Kind::Ident, Tok::Comma => Kind::Comma,
    Tok::Semi => Kind::Semi, Tok::Plus => Kind::Plus, Tok::Star => Kind::Star,
    Tok::LParen => Kind::LParen, Tok::RParen => Kind::RParen,
    Tok::LBracket => Kind::LBracket, Tok::RBracket => Kind::RBracket } }
  fn is_trivia(&self) -> bool { false }
}
impl PunctuatorToken<'_> for Tok {
  fn comma() -> Option<Kind> { Some(Kind::Comma) }
  fn semicolon() -> Option<Kind> { Some(Kind::Semi) }
  fn open_paren() -> Option<Kind> { Some(Kind::LParen) }
  fn close_paren() -> Option<Kind> { Some(Kind::RParen) }
  fn open_bracket() -> Option<Kind> { Some(Kind::LBracket) }
  fn close_bracket() -> Option<Kind> { Some(Kind::RBracket) }
}
impl From<Comma<(), (), ()>> for Kind { fn from(_: Comma<(), (), ()>) -> Self { Kind::Comma } }
impl From<Semicolon<(), (), ()>> for Kind { fn from(_: Semicolon<(), (), ()>) -> Self { Kind::Semi } }
impl From<OpenParen<(), (), ()>> for Kind { fn from(_: OpenParen<(), (), ()>) -> Self { Kind::LParen } }
impl From<CloseParen<(), (), ()>> for Kind { fn from(_: CloseParen<(), (), ()>) -> Self { Kind::RParen } }
impl From<OpenBracket<(), (), ()>> for Kind { fn from(_: OpenBracket<(), (), ()>) -> Self { Kind::LBracket } }
impl From<CloseBracket<(), (), ()>> for Kind { fn from(_: CloseBracket<(), (), ()>) -> Self { Kind::RBracket } }
struct CharLexer<'a> { src: &'a str, pos: usize, tok: SimpleSpan, state: () }
impl<'a> Lexer<'a> for CharLexer<'a> {
  type State = (); type Source = str; type Token = Tok; type Span = SimpleSpan; type Offset = usize;
  fn new(src: &'a str) -> Self { Self { src, pos: 0, tok: SimpleSpan::new(0, 0), state: () } }
  fn with_state(src: &'a str, _: ()) -> Self { Self::new(src) }
  fn check(&self) -> Result<(), Infallible> { Ok(()) }
  fn state(&self) -> &() { &self.state }
  fn state_mut(&mut self) -> &mut () { &mut self.state }
  fn into_state(self) -> Self::State {}
  fn source(&self) -> &'a str { self.src }
  fn span(&self) -> SimpleSpan { self.tok }
  fn slice(&self) -> &'a str { &self.src[self.tok.start()..self.tok.end()] }
  fn lex(&mut self) -> Option<Result<Tok, Infallible>> {
    let bytes = self.src.as_bytes();
    while self.pos < bytes.len() && bytes[self.pos] == b' ' { self.pos += 1; }
    if self.pos >= bytes.len() { return None; }
    let (start, c) = (self.pos, bytes[self.pos] as char);
    self.pos += 1;
    self.tok = SimpleSpan::new(start, self.pos);
    Some(Ok(match c {
      '0'..='9' => Tok::Digit(c as u32 - '0' as u32),
      ',' => Tok::Comma, ';' => Tok::Semi, '+' => Tok::Plus, '*' => Tok::Star,
      '(' => Tok::LParen, ')' => Tok::RParen, '[' => Tok::LBracket, ']' => Tok::RBracket,
      c => Tok::Ident(c),
    }))
  }
  fn read_frontier(&self) -> tokora::ReadFrontier<usize> { tokora::ReadFrontier::SpanEnd }
  fn bump(&mut self, n: &usize) { self.pos += n; }
}
type Ctx<'a> = FatalContext<'a, CharLexer<'a>, Error>;
use tokora::{Parse, ParseInput as _, Parser, labelled};

fn digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    match inp.next()? { Some(sp) => match sp.into_data() { Tok::Digit(n) => Ok(n), _ => Err(Error) }, None => Err(Error) }
}

// `recover`: on failure, rewind and fall back to a default (0).
fn digit_or_zero<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    digit
        .recover(|_inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>, _err: Error| Ok(0u32))
        .parse_input(inp)
}
assert_eq!(Parser::with_parser(digit_or_zero).parse_str("7").unwrap(), 7);
assert_eq!(Parser::with_parser(digit_or_zero).parse_str("+").unwrap(), 0); // `+` isn't a digit → recover

// `labelled`: a diagnostic context (a no-op under the fail-fast `Fatal` emitter).
fn labelled_digit<'a>(inp: &mut InputRef<'a, '_, CharLexer<'a>, Ctx<'a>>) -> Result<u32, Error> {
    labelled("a digit", digit).parse_input(inp)
}
assert_eq!(Parser::with_parser(labelled_digit).parse_str("4").unwrap(), 4);

Error taxonomy

Errors are organized by category under [crate::error]. A parser never dictates a concrete error type: it emits the leaf types below, and your error enum absorbs the ones it uses via From (exactly the impl From<…> for Error block hidden in every example above). Each carries a span. Four of them — UnexpectedTokenOf, MissingTokenOf, SeparatedErrorOf, MissingSyntaxOf — also have an *Of<'inp, L, Lang> alias that projects the lexer’s associated types; the rest you spell with their own parameters (the errors reference says which and why). ErrorOf<'inp, L, Ctx, Lang> is the shorthand for a context’s error type.

The whole taxonomy and the emitter/context surface are covered in depth in the errors, emitters & context reference; chapter 7 is the guided tutorial.

Feature matrix

The default is std and combinators — a plain dependency line sees every family below. docs.rs builds all-features. This is the combinator half of the table; the vocabulary reference carries the full one, backends and tooling included.

The combinator gates

combinators is an umbrella over thirteen family gates, each independently settable so a default-features = false build compiles only the combinators it calls. Everything the families sit on stays unconditional — Parser/Parse/parse*, the ParseInput / TryParseInput / ParseChoice traits, and the substrate combinators (expect, delimited, recover, select, opt, padded, node, labelled, …) — so the gates below turn off families, never the spine.

Where a combinator needs a capability, its where-clause names it — e.g. the many/ builder’s count checks want TooFewEmitter / TooManyEmitter on the emitter. Over a ComposableParseContext (any context whose emitter is a ComposableEmitter and whose error absorbs the five token-level conversions — including the built-in Fatal/Verbose/Silent over a sufficient error type) the default-policy family is available at once, and so are the conversions. A count or separator policy (at_most / bounded / require_leading / require_trailing) needs the wider tier, PolicyParseContext, which carries the same bound plus TooManyEmitter and the missing-separator pair.