Commit f2c01056 authored by Maxime Perrotin's avatar Maxime Perrotin
Browse files

Work around antlr bug to detect end of rule

Antlr stops parsing after the rule is completed, meaning that there may
be some syntax erros that could not be unit tested
e.g. task x := 5;o
the "o" would not be detected as a syntax error if the "task" rule is
completely parsed. the EOF token should help, however it does not: when
using the parser in interactive mode it does complain, but not in
standard parsing..
The workaround is to check the index of a possible following token after
parsing the rule. If it is not none, it means that the buffer still
contains something, which therefore is a syntax error.
parent d912df9e
#!/usr/bin/env python
from opengeode.ogParser import parser_init, antlr3, sdl92Parser, lexer
from antlr3.tree import *
# return a string corresponding to a token number:
token = lambda num: lexer.tokenNamesMap[num]
def test_various_1():
''' Expression (should trigger error after digit 5) '''
test = parser_init(string='''5(o)''')
token_str = test.getTokenStream()
res = test.expression()
print(res.tree.toStringTree())
# Check if there are more tokens AFTER what was parsed
moreToken = test.getTokenStream().LT(1).start
assert moreToken != None
def test_various_2():
''' Detect the syntax error '''
test = parser_init(string=
'''task i := 5;o ''')
res=test.task()
print(res.tree.toStringTree())
# Check if there are more tokens AFTER what was parsed
moreToken = test.getTokenStream().LT(1).start
assert moreToken != None
if __name__ == '__main__':
for name, value in dict(globals()).viewitems():
if name.startswith('test_'):
print('---- Executing {} ----'.format(name))
value()
print('---- Done - {} ----\n'.format(name))
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment