1. 15 Nov, 2016 2 commits
  2. 02 Nov, 2016 1 commit
    • Colin Hogben's avatar
      py: Fix wrong assumption that m_renew will not move if shrinking · f9b6b37c
      Colin Hogben authored
      In both parse.c and qstr.c, an internal chunking allocator tidies up
      by calling m_renew to shrink an allocated chunk to the size used, and
      assumes that the chunk will not move.  However, when MICROPY_ENABLE_GC
      is false, m_renew calls the system realloc, which does not guarantee
      this behaviour.  Environments where realloc may return a different
      pointer include:
      
      (1) mbed-os with MBED_HEAP_STATS_ENABLED (which adds a wrapper around
      malloc & friends; this is where I was hit by the bug);
      
      (2) valgrind on linux (how I diagnosed it).
      
      The fix is to call m_renew_maybe with allow_move=false.
      f9b6b37c
  3. 23 Sep, 2016 1 commit
  4. 06 Jun, 2016 1 commit
  5. 20 May, 2016 1 commit
  6. 10 May, 2016 1 commit
  7. 14 Apr, 2016 1 commit
    • Damien George's avatar
      py: Simplify "and" action within parser by making ident-rules explicit. · 0c1de1cd
      Damien George authored
      Most grammar rules can optimise to the identity if they only have a single
      argument, saving a lot of RAM building the parse tree.  Previous to this
      patch, whether a given grammar rule could be optimised was defined (mostly
      implicitly) by a complicated set of logic rules.  With this patch the
      definition is always specified explicitly by using "and_ident" in the rule
      definition in the grammar.  This simplifies the logic of the parser,
      making it a bit smaller and faster.  RAM usage in unaffected.
      0c1de1cd
  8. 13 Apr, 2016 1 commit
  9. 19 Mar, 2016 1 commit
  10. 23 Feb, 2016 1 commit
  11. 12 Jan, 2016 1 commit
  12. 08 Jan, 2016 1 commit
  13. 07 Jan, 2016 2 commits
    • Damien George's avatar
      py/parse: Improve constant folding to operate on small and big ints. · 22b22650
      Damien George authored
      Constant folding in the parser can now operate on big ints, whatever
      their representation.  This is now possible because the parser can create
      parse nodes holding arbitrary objects.  For the case of small ints the
      folding is still efficient in RAM because the folded small int is stored
      inplace in the parse node.
      
      Adds 48 bytes to code size on Thumb2 architecture.  Helps reduce heap
      usage because more constants can be computed at compile time, leading to
      a smaller parse tree, and most importantly means that the constants don't
      have to be computed at runtime (perhaps more than once).  Parser will now
      be a little slower when folding due to calls to runtime to do the
      arithmetic.
      22b22650
    • Damien George's avatar
      py/parse: Optimise away parse node that's just parenthesis around expr. · 93b37262
      Damien George authored
      Before this patch, (x+y)*z would be parsed to a tree that contained a
      redundant identity parse node corresponding to the parenthesis.  With
      this patch such nodes are optimised away, which reduces memory
      requirements for expressions with parenthesis, and simplifies the
      compiler because it doesn't need to handle this identity case.
      
      A parenthesis parse node is still needed for tuples.
      93b37262
  14. 18 Dec, 2015 1 commit
    • Damien George's avatar
      py: Add MICROPY_ENABLE_COMPILER and MICROPY_PY_BUILTINS_EVAL_EXEC opts. · dd5353a4
      Damien George authored
      MICROPY_ENABLE_COMPILER can be used to enable/disable the entire compiler,
      which is useful when only loading of pre-compiled bytecode is supported.
      It is enabled by default.
      
      MICROPY_PY_BUILTINS_EVAL_EXEC controls support of eval and exec builtin
      functions.  By default they are only included if MICROPY_ENABLE_COMPILER
      is enabled.
      
      Disabling both options saves about 40k of code size on 32-bit x86.
      dd5353a4
  15. 17 Dec, 2015 1 commit
  16. 29 Nov, 2015 3 commits
  17. 17 Nov, 2015 1 commit
  18. 12 Oct, 2015 2 commits
  19. 08 Oct, 2015 1 commit
  20. 01 Oct, 2015 1 commit
  21. 17 Aug, 2015 1 commit
    • Damien George's avatar
      unix-cpy: Remove unix-cpy. It's no longer needed. · 65dc960e
      Damien George authored
      unix-cpy was originally written to get semantic equivalent with CPython
      without writing functional tests.  When writing the initial
      implementation of uPy it was a long way between lexer and functional
      tests, so the half-way test was to make sure that the bytecode was
      correct.  The idea was that if the uPy bytecode matched CPython 1-1 then
      uPy would be proper Python if the bytecodes acted correctly.  And having
      matching bytecode meant that it was less likely to miss some deep
      subtlety in the Python semantics that would require an architectural
      change later on.
      
      But that is all history and it no longer makes sense to retain the
      ability to output CPython bytecode, because:
      
      1. It outputs CPython 3.3 compatible bytecode.  CPython's bytecode
      changes from version to version, and seems to have changed quite a bit
      in 3.5.  There's no point in changing the bytecode output to match
      CPython anymore.
      
      2. uPy and CPy do different optimisations to the bytecode which makes it
      harder to match.
      
      3. The bytecode tests are not run.  They were never part of Travis and
      are not run locally anymore.
      
      4. The EMIT_CPYTHON option needs a lot of extra source code which adds
      heaps of noise, especially in compile.c.
      
      5. Now that there is an extensive test suite (which tests functionality)
      there is no need to match the bytecode.  Some very subtle behaviour is
      tested with the test suite and passing these tests is a much better
      way to stay Python-language compliant, rather than trying to match
      CPy bytecode.
      65dc960e
  22. 24 Jul, 2015 2 commits
  23. 14 Jul, 2015 1 commit
    • Damien George's avatar
      py: Improve allocation policy of qstr data. · ade9a052
      Damien George authored
      Previous to this patch all interned strings lived in their own malloc'd
      chunk.  On average this wastes N/2 bytes per interned string, where N is
      the number-of-bytes for a quanta of the memory allocator (16 bytes on 32
      bit archs).
      
      With this patch interned strings are concatenated into the same malloc'd
      chunk when possible.  Such chunks are enlarged inplace when possible,
      and shrunk to fit when a new chunk is needed.
      
      RAM savings with this patch are highly varied, but should always show an
      improvement (unless only 3 or 4 strings are interned).  New version
      typically uses about 70% of previous memory for the qstr data, and can
      lead to savings of around 10% of total memory footprint of a running
      script.
      
      Costs about 120 bytes code size on Thumb2 archs (depends on how many
      calls to gc_realloc are made).
      ade9a052
  24. 21 Apr, 2015 1 commit
  25. 23 Feb, 2015 1 commit
  26. 13 Feb, 2015 1 commit
  27. 08 Feb, 2015 2 commits
  28. 07 Feb, 2015 1 commit
    • Damien George's avatar
      py: Protect mp_parse and mp_compile with nlr push/pop block. · 0bfc7638
      Damien George authored
      To enable parsing constants more efficiently, mp_parse should be allowed
      to raise an exception, and mp_compile can already raise a MemoryError.
      So these functions need to be protected by an nlr push/pop block.
      
      This patch adds that feature in all places.  This allows to simplify how
      mp_parse and mp_compile are called: they now raise an exception if they
      have an error and so explicit checking is not needed anymore.
      0bfc7638
  29. 24 Jan, 2015 1 commit
  30. 20 Jan, 2015 1 commit
  31. 16 Jan, 2015 1 commit
  32. 14 Jan, 2015 1 commit
  33. 13 Jan, 2015 1 commit
    • Damien George's avatar
      py: Never intern data of large string/bytes object; add relevant tests. · 4c81ba80
      Damien George authored
      Previously to this patch all constant string/bytes objects were
      interned by the compiler, and this lead to crashes when the qstr was too
      long (noticeable now that qstr length storage defaults to 1 byte).
      
      With this patch, long string/bytes objects are never interned, and are
      referenced directly as constant objects within generated code using
      load_const_obj.
      4c81ba80