Info Catalog bovine: Bovine Grammar Rules bovine: Grammar-to-Lisp Details

bovine: How Lexical Tokens Match

 
 2.1 How Lexical Tokens Match
 ============================
 
 A lexical rule must be used to define how to match a lexical token.
 
    For instance:
 
      %keyword FOO "foo"
 
    Means that ‘FOO’ is a reserved language keyword, matched as such by
 looking up into a keyword table, (grammar-fw)keyword Decl.  This
 is because ‘"foo"’ will be converted to ‘FOO’ in the lexical analysis
 stage.  Thus the symbol ‘FOO’ won’t be available any other way.
 
    If we specify our token in this way:
 
      %token <symbol> FOO "foo"
 
    then ‘FOO’ will match the string ‘"foo"’ explicitly, but it won’t do
 so at the lexical level, allowing use of the text ‘"foo"’ in other forms
 of regular expressions.
 
    In that case, ‘FOO’ is a ‘symbol’-type token.  To match, a ‘symbol’
 must first be encountered, and then it must ‘string-match "foo"’.
 
 *Caution:*
      Be especially careful to remember that ‘"foo"’, and more generally
      the %token’s match-value string, is a regular expression!
 
    Non symbol tokens are also allowed.  For example:
 
      %token <punctuation> PERIOD "[.]"
 
      filename : symbol PERIOD symbol
               ;
 
    ‘PERIOD’ is a ‘punctuation’-type token that will explicitly match one
 period when used in the above rule.
 
 *Please Note:*
      ‘symbol’, ‘punctuation’, etc., are predefined lexical token types,
      based on the “syntax class”-character associations currently in
      effect.
 

Info Catalog bovine: Bovine Grammar Rules bovine: Grammar-to-Lisp Details