Token vs. rule

In addition, a rule differs from a token in that whitespace in a rule definition corresponds to calls to a special <?ws> subrule.

The default <?ws> subrule eats up any optional whitespace, or requires whitespace between two word characters.

(The leading '?' in <?ws> indicates that it's a "non-capturing" subrule.)