The lexical grammar of Java has a special case for the the
> character. Normally, tokens are formed based on the longest-match rule, so that an input string of
>>> would form a single token for a bitshift operator. However, this would result in an incorrect parse for types like
List<List<String>> where the two consecutive
> characters should be treated as separate tokens. According to the JLS (§3.5):
If the input character
>appears in a type context (§4.11), that is, as part of a Type or an UnannType in the syntactic grammar (§4.1, §8.3), it is always reduced to the numerical comparison operator
>, even when it could be combined with an adjacent
>character to form a different operator.
Read literally, this means the lexer would need to know the grammatical context in order to decide which tokens to emit; but this context is normally not determined until parsing, after lexing is complete.
Of course, the specification only defines the rules for what source texts form valid Java programs and what those programs mean; a Java compiler could implement the spec in various different ways. But this kind of special case seems difficult to handle without hand-crafting an ad-hoc solution, whereas hand-crafted hacks presumably aren't applicable when using parser generators or other frameworks based on formal grammars. So, how is this special case implemented in real parsers?
While this question is written about Java, the same ambiguity could occur in other languages which share Java's generic syntax, so answers about parsers for other languages are also welcome.