Friday, 9 August 2013

ANTLR4- dynamically inject token

ANTLR4- dynamically inject token

So I'm writing a python parser and I need to dynamically generate INDENT
and DEDENT tokens (because python doesn't use explicit delimiters)
according to the python grammar specification.
Basically I have a stack of integers representing indentation levels. In
an embedded java action in the INDENT token, I check if the current level
of indentation is higher than the level on top of the stack; if it is, I
push it on; if not, I call skip().
The problem is, if the current indentation level matches a level multiple
levels down in the stack, I have to generate multiple DEDENT tokens, and I
can't figure out how to do that.
My current code: (note that within_indent_block and current_indent_level
are managed elsewhere)
fragment DENT: {within_indent_block}? (SPACE|TAB)+;
INDENT: {within_indent_block}? DENT
{if(current_indent_level > whitespace_stack.peek().intValue()){
whitespace_stack.push(new Integer(current_indent_level));
within_indent_block = false;
}else{
skip();
}
}
;
DEDENT: {within_indent_block}? DENT
{if(current_indent_level < whitespace_stack.peek().intValue()){
while(current_indent_level < whitespace_stack.peek().intValue()){
whitespace_stack.pop();
<<injectDedentToken()>>; //how do I do this
}
}else{
skip();
}
}
;
How do I do this and / or is there a better way?

No comments:

Post a Comment