21.2 Overriding The Default Memory Management

Again, this section applies only to languages with manual memory management.

Flex calls the functions yyalloc, yyrealloc, and yyfree when it needs to allocate or free memory. By default, these functions are wrappers around the standard C functions, malloc, realloc, and free, respectively. You can override the default implementations by telling flex that you will provide your own implementations.

To override the default implementations, you must do two things:

1. Suppress the default implementations by specifying one or more of the following options:
   • %option noyyalloc
   • %option noyyrealloc
   • %option noyyfree.
2. Provide your own implementation of the following functions: ³

   // For a non-reentrant scanner
   void * yyalloc (size_t bytes);
   void * yyrealloc (void * ptr, size_t bytes);
   void   yyfree (void * ptr);
   
   // For a reentrant scanner
   void * yyalloc (size_t bytes, void * yyscanner);
   void * yyrealloc (void * ptr, size_t bytes, void * yyscanner);
   void   yyfree (void * ptr, void * yyscanner);
   

In the following example, we will override all three memory routines. We assume that there is a custom allocator with garbage collection. In order to make this example interesting, we will use a reentrant scanner, passing a pointer to the custom allocator through yyextra.

%{
#include "some_allocator.h"
%}

/* Suppress the default implementations. */
%option noyyalloc noyyrealloc noyyfree
%option reentrant
%option extra-type="struct allocator*"

/* Initialize the allocator. */
%option user-init="yyextra = allocator_create();"
%%
.|\n   ;
%%

/* Provide our own implementations. */
void * yyalloc (size_t bytes, void* yyscanner) {
    return allocator_alloc (yyextra, bytes);
}

void * yyrealloc (void * ptr, size_t bytes, void* yyscanner) {
    return allocator_realloc (yyextra, bytes);
}

void yyfree (void * ptr, void * yyscanner) {      
    /* Do nothing -- we leave it to the garbage collector. */
}