16.2 Options Affecting Scanner Behavior

‘-i, --case-insensitive, %option case-insensitive’

instructs flex to generate a case-insensitive scanner. The case of letters given in the flex input patterns will be ignored, and tokens in the input will be matched regardless of case. The matched text given in yytext will have the preserved case (i.e., it will not be folded). For tricky behavior, see case and character ranges.

‘-l, --lex-compat, %option lex-compat’

turns on maximum compatibility with the original AT&T lex implementation. Note that this does not mean full compatibility. Use of this option costs a considerable amount of performance, and it cannot be used with the ‘--c++’, ‘--full’, ‘--fast’, ‘-Cf’, or ‘-CF’ options. For details on the compatibilities it provides, see Lex and Posix. This option also results in the name YY_FLEX_LEX_COMPAT being #define’d in the generated scanner.

‘-B, --batch, %option batch’

instructs flex to generate a batch scanner, the opposite of interactive scanners generated by ‘--interactive’ (see below). In general, you use ‘-B’ when you are certain that your scanner will never be used interactively, and you want to squeeze a little more performance out of it. If your goal is instead to squeeze out a lot more performance, you should be using the ‘-Cf’ or ‘-CF’ options, which turn on ‘--batch’ automatically anyway.

‘-I, --interactive, %option interactive’

instructs flex to generate an interactive scanner. An interactive scanner is one that only looks ahead to decide what token has been matched if it absolutely must. It turns out that always looking one extra character ahead, even if the scanner has already seen enough text to disambiguate the current token, is a bit faster than only looking ahead when necessary. But scanners that always look ahead give dreadful interactive performance; for example, when a user types a newline, it is not recognized as a newline token until they enter another token, which often means typing in another whole line.

flex scanners default to interactive unless you use the ‘-Cf’ or ‘-CF’ table-compression options (see Performance). That’s because if you’re looking for high-performance you should be using one of these options, so if you didn’t, flex assumes you’d rather trade off a bit of run-time performance for intuitive interactive behavior. Note also that you cannot use ‘--interactive’ in conjunction with ‘-Cf’ or ‘-CF’. Thus, this option is not really needed; it is on by default for all those cases in which it is allowed.

You can force a scanner to not be interactive by using ‘--batch’

‘-7, --7bit, %option 7bit’

instructs flex to generate a 7-bit scanner, i.e., one which can only recognize 7-bit characters in its input. The advantage of using ‘--7bit’ is that the scanner’s tables can be up to half the size of those generated using the ‘--8bit’. The disadvantage is that such scanners often hang or crash if their input contains an 8-bit character.

Note, however, that unless you generate your scanner using the ‘-Cf’ or ‘-CF’ table compression options, use of ‘--7bit’ will save only a small amount of table space, and make your scanner considerably less portable. Flex’s default behavior is to generate an 8-bit scanner unless you use the ‘-Cf’ or ‘-CF’, in which case flex defaults to generating 7-bit scanners unless your site was always configured to generate 8-bit scanners (as will often be the case with non-USA sites). You can tell whether flex generated a 7-bit or an 8-bit scanner by inspecting the flag summary in the ‘--verbose’ output as described above.

Note that if you use ‘-Cfe’ or ‘-CFe’ flex still defaults to generating an 8-bit scanner, since usually with these compression options full 8-bit tables are not much more expensive than 7-bit tables.

‘-8, --8bit, %option 8bit’

instructs flex to generate an 8-bit scanner, i.e., one which can recognize 8-bit characters. This flag is only needed for scanners generated using ‘-Cf’ or ‘-CF’, as otherwise flex defaults to generating an 8-bit scanner anyway.

See the discussion of ‘--7bit’ above for flex’s default behavior and the tradeoffs between 7-bit and 8-bit scanners.

‘--default, %option default’

generate the default rule.

‘--always-interactive, %option always-interactive’

instructs flex to generate a scanner which always considers its input interactive. Normally, on each new input file the scanner calls isatty() in an attempt to determine whether the scanner’s input source is interactive and thus should be read a character at a time. When this option is used, however, then no such call is made.

‘--never-interactive, --never-interactive’

instructs flex to generate a scanner which never considers its input interactive. This is the opposite of always-interactive.

‘-X, --posix, %option posix’

turns on maximum compatibility with the POSIX 1003.2-1992 definition of lex. Since flex was originally designed to implement the POSIX definition of lex this generally involves very few changes in behavior. At the current writing the known differences between flex and the POSIX standard are:

• In POSIX and AT&T lex, the repeat operator, ‘{}’, has lower precedence than concatenation (thus ‘ab{3}’ yields ‘ababab’). Most POSIX utilities use an Extended Regular Expression (ERE) precedence that has the precedence of the repeat operator higher than concatenation (which causes ‘ab{3}’ to yield ‘abbb’). By default, flex places the precedence of the repeat operator higher than concatenation which matches the ERE processing of other POSIX utilities. When either ‘--posix’ or ‘-l’ are specified, flex will use the traditional AT&T and POSIX-compliant precedence for the repeat operator where concatenation has higher precedence than the repeat operator.

‘--stack, %option stack’

enables the use of start condition stacks (see Start Conditions).

‘--stdinit, %option stdinit’

if set (i.e., %option stdinit) initializes yyin and yyout to stdin and stdout, instead of the default of NULL. Some existing lex programs depend on this behavior, even though it is not compliant with ANSI C, which does not require stdin and stdout to be compile-time constant. In a reentrant scanner, however, this is not a problem since initialization is performed in yylex_init at runtime.

‘--yylineno, %option yylineno’

directs flex to generate a scanner that maintains the number of the current line read from its input in the global variable yylineno. This option is implied by %option lex-compat. In a reentrant C scanner, the macro yylineno is accessible regardless of the value of %option yylineno, however, its value is not modified by flex unless %option yylineno is enabled.

‘--yywrap, %option yywrap’

if unset (i.e., --noyywrap), makes the scanner not call yywrap() upon an end-of-file, but simply assume that there are no more files to scan (until the user points yyin at a new file and calls yylex() again).