Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Sign in
Toggle navigation
Menu
Open sidebar
TASTE
taste-setup
Commits
d6aa8950
Commit
d6aa8950
authored
Feb 21, 2017
by
Thanassis Tsiodras
Browse files
ANTLR3 runtime
parent
99e41047
Changes
26
Expand all
Hide whitespace changes
Inline
Side-by-side
antlr/antlr_python_runtime-3.1.3/AUTHORS
0 → 100644
View file @
d6aa8950
Benjamin Niemann <pink at odahoda dot de>: Main developer of Python target.
Clinton Roy <clinton.roy at gmail dot com>: AST templates and runtime.
antlr/antlr_python_runtime-3.1.3/LICENSE
0 → 100644
View file @
d6aa8950
[The "BSD licence"]
Copyright (c) 2003-2006 Terence Parr
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. The name of the author may not be used to endorse or promote products
derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
antlr/antlr_python_runtime-3.1.3/MANIFEST.in
0 → 100644
View file @
d6aa8950
include LICENSE AUTHORS ez_setup.py
antlr/antlr_python_runtime-3.1.3/PKG-INFO
0 → 100644
View file @
d6aa8950
Metadata-Version: 1.0
Name: antlr_python_runtime
Version: 3.1.3
Summary: Runtime package for ANTLR3
Home-page: http://www.antlr.org/
Author: Benjamin Niemann
Author-email: pink@odahoda.de
License: BSD
Download-URL: http://www.antlr.org/download.html
Description: This is the runtime package for ANTLR3, which is required to use parsers
generated by ANTLR3.
Platform: UNKNOWN
antlr/antlr_python_runtime-3.1.3/README
0 → 100644
View file @
d6aa8950
1) ABOUT
========
This is the Python package 'antlr3', which is required to use parsers created
by the ANTLR3 tool. See <http://www.antlr.org/> for more information about
ANTLR3.
2) STATUS
=========
The Python target for ANTLR3 is still in beta. Documentation is lacking, some
bits of the code is not yet done, some functionality has not been tested yet.
Also the API might change a bit - it currently mimics the Java implementation,
but it may be made a bit more pythonic here and there.
WARNING: Currently the runtime library for V3.1 is not compatible with
recognizers generated by ANTLR V3.0.x. If you are an application developer,
then the suggested way to solve this is to package the correct runtime with
your application. Installing the runtime in the global site-packages directory
may not be a good idea.
It is still undetermined, if a future release of the V3.1 runtime will be
compatible with V3.0.x recognizers or if future runtimes V3.2+ will be
compatible with V3.1 recognizers.
Sorry for the inconvenience.
3) DOWNLOAD
===========
This runtime is part of the ANTLR distribution. The latest version can be found
at <http://www.antlr.org/download.html>.
If you are interested in the latest, most bleeding edge version, have a look at
the perforce depot at <http://fisheye2.cenqua.com/browse/antlr>. There are
tarballs ready to download, so you don't have to install the perforce client.
4) INSTALLATION
===============
Just like any other Python package:
$ python setup.py install
See <http://docs.python.org/inst/> for more information.
5) DOCUMENTATION
================
Documentation (as far as it exists) can be found in the wiki
<http://www.antlr.org/wiki/display/ANTLR3/Antlr3PythonTarget>
6) REPORTING BUGS
=================
Please send bug reports to the ANTLR mailing list
<http://www.antlr.org:8080/mailman/listinfo/antlr-interest> or
<pink@odahoda.de>.
Existing bugs may appear someday in the bugtracker:
<http://www.antlr.org:8888/browse/ANTLR>
7) HACKING
==========
Only the runtime package can be found here. There are also some StringTemplate
files in 'src/org/antlr/codegen/templates/Python/' and some Java code in
'src/org/antlr/codegen/PythonTarget.java' (of the main ANTLR3 source
distribution).
If there are no directories 'tests' and 'unittests' in 'runtime/Python', you
should fetch the latest ANTLR3 version from the perforce depot. See section
DOWNLOAD.
You'll need java and ant in order to compile and use the tool.
Be sure to properly setup your CLASSPATH.
(FIXME: is there some generic information, how to build it yourself? I should
point to it to avoid duplication.)
You can then use the commands
$ python setup.py unittest
$ python setup.py functest
to ensure that changes do not break existing behaviour.
Please send patches to <pink@odahoda.de>. For larger code contributions you'll
have to sign the "Developer's Certificate of Origin", which can be found on
<http://www.antlr.org/license.html> or use the feedback form at
<http://www.antlr.org/misc/feedback>.
antlr/antlr_python_runtime-3.1.3/antlr3/__init__.py
0 → 100644
View file @
d6aa8950
""" @package antlr3
@brief ANTLR3 runtime package
This module contains all support classes, which are needed to use recognizers
generated by ANTLR3.
@mainpage
\n
ote Please be warned that the line numbers in the API documentation do not
match the real locations in the source code of the package. This is an
unintended artifact of doxygen, which I could only convince to use the
correct module names by concatenating all files from the package into a single
module file...
Here is a little overview over the most commonly used classes provided by
this runtime:
@section recognizers Recognizers
These recognizers are baseclasses for the code which is generated by ANTLR3.
- BaseRecognizer: Base class with common recognizer functionality.
- Lexer: Base class for lexers.
- Parser: Base class for parsers.
- tree.TreeParser: Base class for %tree parser.
@section streams Streams
Each recognizer pulls its input from one of the stream classes below. Streams
handle stuff like buffering, look-ahead and seeking.
A character stream is usually the first element in the pipeline of a typical
ANTLR3 application. It is used as the input for a Lexer.
- ANTLRStringStream: Reads from a string objects. The input should be a unicode
object, or ANTLR3 will have trouble decoding non-ascii data.
- ANTLRFileStream: Opens a file and read the contents, with optional character
decoding.
- ANTLRInputStream: Reads the date from a file-like object, with optional
character decoding.
A Parser needs a TokenStream as input (which in turn is usually fed by a
Lexer):
- CommonTokenStream: A basic and most commonly used TokenStream
implementation.
- TokenRewriteStream: A modification of CommonTokenStream that allows the
stream to be altered (by the Parser). See the 'tweak' example for a usecase.
And tree.TreeParser finally fetches its input from a tree.TreeNodeStream:
- tree.CommonTreeNodeStream: A basic and most commonly used tree.TreeNodeStream
implementation.
@section tokenstrees Tokens and Trees
A Lexer emits Token objects which are usually buffered by a TokenStream. A
Parser can build a Tree, if the output=AST option has been set in the grammar.
The runtime provides these Token implementations:
- CommonToken: A basic and most commonly used Token implementation.
- ClassicToken: A Token object as used in ANTLR 2.x, used to %tree
construction.
Tree objects are wrapper for Token objects.
- tree.CommonTree: A basic and most commonly used Tree implementation.
A tree.TreeAdaptor is used by the parser to create tree.Tree objects for the
input Token objects.
- tree.CommonTreeAdaptor: A basic and most commonly used tree.TreeAdaptor
implementation.
@section Exceptions
RecognitionException are generated, when a recognizer encounters incorrect
or unexpected input.
- RecognitionException
- MismatchedRangeException
- MismatchedSetException
- MismatchedNotSetException
.
- MismatchedTokenException
- MismatchedTreeNodeException
- NoViableAltException
- EarlyExitException
- FailedPredicateException
.
.
A tree.RewriteCardinalityException is raised, when the parsers hits a
cardinality mismatch during AST construction. Although this is basically a
bug in your grammar, it can only be detected at runtime.
- tree.RewriteCardinalityException
- tree.RewriteEarlyExitException
- tree.RewriteEmptyStreamException
.
.
"""
# tree.RewriteRuleElementStream
# tree.RewriteRuleSubtreeStream
# tree.RewriteRuleTokenStream
# CharStream
# DFA
# TokenSource
# [The "BSD licence"]
# Copyright (c) 2005-2008 Terence Parr
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. The name of the author may not be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
__version__
=
'3.1.3'
def
version_str_to_tuple
(
version_str
):
import
re
import
sys
if
version_str
==
'HEAD'
:
return
(
sys
.
maxint
,
sys
.
maxint
,
sys
.
maxint
,
sys
.
maxint
)
m
=
re
.
match
(
r
'(\d+)\.(\d+)(\.(\d+))?(b(\d+))?'
,
version_str
)
if
m
is
None
:
raise
ValueError
(
"Bad version string %r"
%
version_str
)
major
=
int
(
m
.
group
(
1
))
minor
=
int
(
m
.
group
(
2
))
patch
=
int
(
m
.
group
(
4
)
or
0
)
beta
=
int
(
m
.
group
(
6
)
or
sys
.
maxint
)
return
(
major
,
minor
,
patch
,
beta
)
runtime_version_str
=
__version__
runtime_version
=
version_str_to_tuple
(
runtime_version_str
)
from
constants
import
*
from
dfa
import
*
from
exceptions
import
*
from
recognizers
import
*
from
streams
import
*
from
tokens
import
*
antlr/antlr_python_runtime-3.1.3/antlr3/compat.py
0 → 100644
View file @
d6aa8950
"""Compatibility stuff"""
# begin[licence]
#
# [The "BSD licence"]
# Copyright (c) 2005-2008 Terence Parr
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. The name of the author may not be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# end[licence]
try
:
set
=
set
frozenset
=
frozenset
except
NameError
:
from
sets
import
Set
as
set
,
ImmutableSet
as
frozenset
try
:
reversed
=
reversed
except
NameError
:
def
reversed
(
l
):
l
=
l
[:]
l
.
reverse
()
return
l
antlr/antlr_python_runtime-3.1.3/antlr3/constants.py
0 → 100644
View file @
d6aa8950
"""ANTLR3 runtime package"""
# begin[licence]
#
# [The "BSD licence"]
# Copyright (c) 2005-2008 Terence Parr
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. The name of the author may not be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# end[licence]
EOF
=
-
1
## All tokens go to the parser (unless skip() is called in that rule)
# on a particular "channel". The parser tunes to a particular channel
# so that whitespace etc... can go to the parser on a "hidden" channel.
DEFAULT_CHANNEL
=
0
## Anything on different channel than DEFAULT_CHANNEL is not parsed
# by parser.
HIDDEN_CHANNEL
=
99
# Predefined token types
EOR_TOKEN_TYPE
=
1
##
# imaginary tree navigation type; traverse "get child" link
DOWN
=
2
##
#imaginary tree navigation type; finish with a child list
UP
=
3
MIN_TOKEN_TYPE
=
UP
+
1
INVALID_TOKEN_TYPE
=
0
antlr/antlr_python_runtime-3.1.3/antlr3/debug.py
0 → 100644
View file @
d6aa8950
This diff is collapsed.
Click to expand it.
antlr/antlr_python_runtime-3.1.3/antlr3/dfa.py
0 → 100644
View file @
d6aa8950
"""ANTLR3 runtime package"""
# begin[licence]
#
# [The "BSD licence"]
# Copyright (c) 2005-2008 Terence Parr
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# 1. Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# 2. Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# 3. The name of the author may not be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
# IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
# OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
# NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# end[licensc]
from
antlr3.constants
import
EOF
from
antlr3.exceptions
import
NoViableAltException
,
BacktrackingFailed
class
DFA
(
object
):
"""@brief A DFA implemented as a set of transition tables.
Any state that has a semantic predicate edge is special; those states
are generated with if-then-else structures in a specialStateTransition()
which is generated by cyclicDFA template.
"""
def
__init__
(
self
,
recognizer
,
decisionNumber
,
eot
,
eof
,
min
,
max
,
accept
,
special
,
transition
):
## Which recognizer encloses this DFA? Needed to check backtracking
self
.
recognizer
=
recognizer
self
.
decisionNumber
=
decisionNumber
self
.
eot
=
eot
self
.
eof
=
eof
self
.
min
=
min
self
.
max
=
max
self
.
accept
=
accept
self
.
special
=
special
self
.
transition
=
transition
def
predict
(
self
,
input
):
"""
From the input stream, predict what alternative will succeed
using this DFA (representing the covering regular approximation
to the underlying CFL). Return an alternative number 1..n. Throw
an exception upon error.
"""
mark
=
input
.
mark
()
s
=
0
# we always start at s0
try
:
for
_
in
xrange
(
50000
):
#print "***Current state = %d" % s
specialState
=
self
.
special
[
s
]
if
specialState
>=
0
:
#print "is special"
s
=
self
.
specialStateTransition
(
specialState
,
input
)
if
s
==
-
1
:
self
.
noViableAlt
(
s
,
input
)
return
0
input
.
consume
()
continue
if
self
.
accept
[
s
]
>=
1
:
#print "accept state for alt %d" % self.accept[s]
return
self
.
accept
[
s
]
# look for a normal char transition
c
=
input
.
LA
(
1
)
#print "LA = %d (%r)" % (c, unichr(c) if c >= 0 else 'EOF')
#print "range = %d..%d" % (self.min[s], self.max[s])
if
c
>=
self
.
min
[
s
]
and
c
<=
self
.
max
[
s
]:
# move to next state
snext
=
self
.
transition
[
s
][
c
-
self
.
min
[
s
]]
#print "in range, next state = %d" % snext
if
snext
<
0
:
#print "not a normal transition"
# was in range but not a normal transition
# must check EOT, which is like the else clause.
# eot[s]>=0 indicates that an EOT edge goes to another
# state.
if
self
.
eot
[
s
]
>=
0
:
# EOT Transition to accept state?
#print "EOT trans to accept state %d" % self.eot[s]
s
=
self
.
eot
[
s
]
input
.
consume
()
# TODO: I had this as return accept[eot[s]]
# which assumed here that the EOT edge always
# went to an accept...faster to do this, but
# what about predicated edges coming from EOT
# target?
continue
#print "no viable alt"
self
.
noViableAlt
(
s
,
input
)
return
0
s
=
snext
input
.
consume
()
continue
if
self
.
eot
[
s
]
>=
0
:
#print "EOT to %d" % self.eot[s]
s
=
self
.
eot
[
s
]
input
.
consume
()
continue
# EOF Transition to accept state?
if
c
==
EOF
and
self
.
eof
[
s
]
>=
0
:
#print "EOF Transition to accept state %d" \
# % self.accept[self.eof[s]]
return
self
.
accept
[
self
.
eof
[
s
]]
# not in range and not EOF/EOT, must be invalid symbol
self
.
noViableAlt
(
s
,
input
)
return
0
else
:
raise
RuntimeError
(
"DFA bang!"
)
finally
:
input
.
rewind
(
mark
)
def
noViableAlt
(
self
,
s
,
input
):
if
self
.
recognizer
.
_state
.
backtracking
>
0
:
raise
BacktrackingFailed
nvae
=
NoViableAltException
(
self
.
getDescription
(),
self
.
decisionNumber
,
s
,
input
)
self
.
error
(
nvae
)
raise
nvae
def
error
(
self
,
nvae
):
"""A hook for debugging interface"""
pass
def
specialStateTransition
(
self
,
s
,
input
):
return
-
1
def
getDescription
(
self
):
return
"n/a"
## def specialTransition(self, state, symbol):
## return 0
def
unpack
(
cls
,
string
):
"""@brief Unpack the runlength encoded table data.
Terence implemented packed table initializers, because Java has a
size restriction on .class files and the lookup tables can grow
pretty large. The generated JavaLexer.java of the Java.g example
would be about 15MB with uncompressed array initializers.
Python does not have any size restrictions, but the compilation of
such large source files seems to be pretty memory hungry. The memory