cDRhgXj0

1# Review Notes on Changes to app/lua53
2
3These are my notes following my own review of the lua53 commit diff file:
4
5## Reasons for functional changes
6
7-  The type `LUA_TTABLE` now has subtypes `LUA_TTBLRAM` and `LUA_TTBLROF`, with handling of these subtypes following the model adopted for strings being split into short and long subtypes. In general the variant coding for table subtypes is managed as low as possible in the `ltable.c` routine.  The new `ROTable` is a cut down version of `Table` which only includes the fields used in ROTables.
8
9-  The `getXXX(o)` access macros replace `(o)->XXX` field accesses for lu_byte fields in records that could be in constant (flash-based) memory.
10
11-  Functional support for dynamically loaded libraries removed from `lauxlib.c`, `ldo.c`, and `loadlib.c`.
12
13-  On firmware builds all ROM modules and base functions are in the ROTable `ROM`.  The metatable of `_G` is `_G` and both `__index` and `ROM` point to this ROTable. On `luac.cross` builds `ROM` only contains the ROM modules, with its meta `__index` pointing to a separate `baselib` ROTable.  This gives the fastest runtime performance in the case of firmware builds, but `luac.cross` builds can now be linked using standard linker defaults without any GCC, MSVC, etc. botches.  In both cases all ROM tables and base functions are resolved through the global environment, `_G`.
14
15-  The new 5.3 string cache is replaced by a unified keycache which is used for both string and ROTable entry caching.  The hash algo is now prime multiplier based to allow the use of a 2^n `KEYCACHE_N` number of rows.  This avoid the need for an expensive modulus call during cache lookup.
16
17-  `LCD` style compressed line info is implemented as standard through `lcode.c` and `ldebug.c`.
18
19-  The `math` library is now pretty complete.
20
21- `ldblib.c` now contains a pretty complete `debug` implementation (less `debug.debug` for firmware builds as this would require take-over of stdin). 
22
23-  Error reporting is not directed to `stderr`, but instead the error string is posted as an upval to the C closure  error reporter helper.  This then calls the Lua error reporter as a separate task. 
24
25- `lapi.c` and `lauxlib.c` implement the API and interface changes listed below.
26
27- `lobject.c` includes a fast implementation of `luaO_ceillog2()` using an `asm("nsau %0, %1;")` instruction which is a _lot_ faster and avoids the need for a byte lookup table. 
28
29-  The `lstate.c` seed aglo is fixed rather than using randomisation. 
30
31-  `lua.c` is a complete reimplementation more closely following the current NodeMCU 5.1 `lua.c` implementation.  This is as a result of architectural drivers arising from its context and being initiated within the startup sequence of the IoT embedded runtime.
32    1.  Processing is based on a single threaded event loop model.  The Lua interactive mode processes input lines from a stdin pipe.  This must be handled on a line by line basis, and other Lua tasks can interleave any multiline processing, so the standard doREPL approach doesn't work.
33    2.  Most OS services and environment processing are supported so much of the standard functionality is irrelevant and is stripped out for simplicity.
34    3.  `stderr` and `stdout` redirection aren't offered as an SDK service, so this is handled in the baselib print function and errors are sent to print.
35
36-  `luaconf.h` has been reworked to reflect the IoT implementation.
37
38-  locales support has been removed from `lvm.c`.  Ditto caching of last closure in Proto records.
39
40-  `ltest.c` has its Memcontrol structure extended to include a double linked list.  This allows me to set H/W watchpoints on dangling blocks in host gdb to work out what blocks aren't being GCed.
41
42-  The test suite has disabled tests which aren't appropriate (e.g. dynamic loading).  Even so I need to do a second pass to make sure that I haven't disabled valid, but failing tests.
43
44
45## API and Interface Changes
46
47-  **lua_gc** has an extra parameter option `LUA_GCSETMEMLIMIT` to set the ECG memory limit.
48
49-  **lua_pushrotable**,  `void lua_pushrotable(lua_State *L, const ROTable *p)` can be used to push a ROtable onto the Lua Stack.
50
51-  **lua_createrotable**,  `void lua_createrotable(lua_State *L, ROTable *t, const ROTable_entry *e, ROTable *mt)`  is used the create a ROTable header for the specified `ROTable_entry` vector.  This is only required for linker-marshalled entry vectors as the ROTable is normally generated by the `LROT` macro declarations.
52
53-  **lua_getstate**, `lua_State *lua_getstate(void)`. Returns the L0 state.  Used in C modules CBs to call the Lua VM.
54
55-  **lua_getcache**, `KeyCache *lua_getcache(int cacheline)` used internally to access the string and table key cache.
56
57-  **lua_getstrings**, `int lua_getstrings(lua_State *L, int opt)` returns a table of strings in the specified string table (0 = RAM, 1 = LFS ROM)
58
59-  **lua_freeheap**, `int lua_freeheap(void)` returns the amount of free heap.
60
61-  **luaL_rometatable**,  `int  luaL_rometatable(lua_State *L, const char* tname, const ROTable *p)` shorthand for `lua_pushrotable()` and `luaL_newmetatable()`; used to associate a ROTable metatable with the entry `tname` in the Lua registry.
62
63-  **luaL_pcallx**,  `int luaL_pcallx(lua_State *L, int narg, int nres)`.  This is designed as a plug-in replacement for `lua_call(L, n, m)` used to call Lua CBs in the module event routines.  Unlike `lua_call()`, this does a protected call and returns a call status.  In the case of the called routine throwing an error, instead of the VM panicing and restarting, the error handler collects a full traceback and posts a separate task with this error string as an upval.  The error reporter then calls the users reporter function, which can then print or log the error, and continue or restart as required.
64
65-  **luaL_posttask**, `int luaL_posttask(lua_State* L, int prio)` Post the task popped from the stack at the specified priority. 
66
67The build will compile in the Lua Test suite if `LUA_ENABLE_TEST` and `LUA_USE_HOST` are defined -- that is for test `luac.cross` builds only. 
68
69## To be implemented
70
71-  `lgc.c` to honour the `lua_gc()` option `LUA_GCSETMEMLIMIT`
72
73-  Replace all `lua_call()` uses in the `app/modules` files with the `luaL_pcallx()` variant. 
74
75-  Support loading both standard (format=0) and NodeMCU (format=10) dump formats.  Also use the head to determin the int and float sizes.
76
77-  LFS implementation
78
79-  Write up a howto on using the `LROT` macros.
80 
81## Known Issues
82
83-  `debug.getstrings()` raises a `E:M 32784 not enough memory` error.