src/compiler/bootstrap
src/compiler/metalua
junk
+patches
ran
Installation guidelines
=======================
-TESTING AN INSTALLATION PROCEDURE IS HARD, AND RARELY DONE RIGHT FROM
-THE FIRST TIME. IF YOU EXPERIENCE INSTALLATION TROUBLES, PLEASE REPORT
+======================================================================
+TESTING AN INSTALLATION PROCEDURE IS HARD, AND RARELY DONE RIGHT AT
+THE FIRST TRY. IF YOU EXPERIENCE INSTALLATION TROUBLES, PLEASE REPORT
THEM, TO AVOID THEM TO FUTURE USERS. mailto:metalua@gmail.com
+======================================================================
Prerequisites
-------------
- edit metalua\src\win32\make-win32-msvc.bat, and set the
user configuration:
- - The directory where you want to put your libs and executables,
+ * The directory where you want to put your libs and executables,
MUALIB_TARGET.
- - The directory where the compiler and linker are,
+ * The directory where the compiler and linker are,
MSVCDIR.
- open a DOS shell in metalua\src, and run the batch file above:
d:\fabien\src\metalua\src> win32\make-win32-msvc
-Notice that under windows, Pluto is linked statically with the Lua VM,
-as it exploits some APIs which are not exported as DLL by default. If
-you want to run your own Lua VM, you'll need either:
-- to set the LUA_MFAST environment variable so that the compiler won't
- use Pluto; but then, interactions between compile-time parts of
- files compiled together might sometimes cause troubles
-- or recompile your VM with Pluto linked in it
-- or recompile it by exporting the LUAI_FUNC functions as DLL entry
- points.
-
- Set your environment variables. A script metalua\src\mlua_setenv.bat
must have been generated, which contains the appropriate
settings. If you want to commit these setting permanently, you'll
-----
There is also a migw port, which hasn't been tested for a while and is
likely to be broken. I'd be grateful if someone provided a patch for
-this. I've never tried to compile it under cygwin either,
-contributions are welcome.
+this. I've never tried to compile it under cygwin either, patches are
+welcome.
Un*x
----
Troubleshooting:
----------------
-
Under Linux, if you don't have readline and don't want to install it
(e.g. you rely on rlwrap or ledit), you'll have to remove -lreadline
from lua/Makefile, and to remove the #define LUA_USE_READLINE from
lua/luaconf.h
If you use your own Lua VM, depending on your OS, you might have
-problems with unexported symbols which are required by Pluto. Either
-link Pluto with statically with your VM (take inspiration from the
-win32 hack above),
-
-
+problems with unexported symbols required by Pluto. Pluto requires the
+symbols prefixed with LUAI_FUNC in the VM to be exported, and they
+aren't by default under Linux and Windows. This is set in luaconf.h;
+you'll need to change that setting in your own Lua VM if you want to
+use it.
+
+Alternatively, you can let metalua run without Pluto by setting the
+environment variable LUA_MFAST to "yes". However, this might cause
+parasitic interactions when several source files are compiled
+simultaneously, so you'd rather precompile each file and library one
+by one.
Test drive
----------
Dynamic Parsers
---------------
-One of these tools is dynamic parser: a source file can change the
-syntax recognized by the parser while it's being parsed. Taken alone,
-this feature lets you make superficial syntax tweaks on the
-language. The parser is based on a parser combinator called 'gg'. You
-should know the half dozen functions in gg API to do advanced things,
-but it means you can use and define functions that transform parsers:
+
+One of the tools is the dynamic parser, which allows a source file to
+change the grammar recognized by the parser, while it's being
+parsed. Taken alone, this feature lets you make superficial syntax
+tweaks on the language. The parser is based on a parser combinator
+library called 'gg'; you should know the half dozen functions in gg
+API to do advanced things:
- There are a couple of very simple combinators like gg.list,
gg.sequence, qq.multisequence, gg.optkeyword etc. that offer a level
mlp.expr parses Lua expressions, gg.list{ mlp.expr } creates a
parser which handles lists of Lua expressions.
-- Since you can create all the combinators you can think of, there
- also are combinators specialized for typical language tasks. In
- Yacc-like systems, the language definition quickly becomes
- unreadable, because all non-native features have to be encoded in
- clumsy and brittle ways; so if your parser won't natively let you
- specify infix operator precedence and associativity easily, tough
- luck for you and your code maintainers. With combinators, this is
- abstracted away in a regular function, so you just write:
+- Since you can create all the combinators you can think of (they're
+ regular, higher-order functions), there also are combinators
+ specialized for typical language tasks. In Yacc-like systems, the
+ language definition quickly becomes unreadable, because all
+ non-native features have to be encoded in clumsy and brittle ways.
+ So if your parser won't natively let you specify infix operator
+ precedence and associativity easily, tough luck for you and your
+ code maintainers. With combinators OTOH, most of such useful
+ functions already exist, and you can write your owns without
+ rewriting the parser itself. For instance, adding an infix operator
+ would just look like:
> mlp.expr.infix:add{ "xor", prec=40, assoc='left', builder=xor_builder }
compiling a valid one, for a use=able compiler.
Yacc-like systems might seem simpler to adopt than combinators, as
-long as they're used on extremely simple problems. However, if if you
+long as they're used on extremely simple problems. However, if you
either try to write something non trivial, or to write a simple macro
in a robust way, you'll need to use lots of messy tricks and hacks,
-and spend much more time getting them (seemingly) ritght than that 1/2
-hour required to master most of gg.
+and spend much more time getting them (approximatively) right than
+that 1/2 hour required to master the regular features of gg.
Real meta-programming
---------------------
+
If you plan to go beyond trivial keyword-for-keyword syntax tweaks,
-what will limit you is the ability to manipulate source code
-conveniently: without the proper tools and abstractions, even the
-simplest tasks will turn into a dirty hacks fest, then either into a
-nightmare, or most often into abandonware. Providing an empowering
-framework is metalua's whole purpose. The core principle is that
-programs prefer to manipulate code as trees (whereas most developers
-prefer ASCII sources). The make-or-break deal is then:
+what will limit you is not syntax definition, but the ability to
+manipulate source code conveniently: without the proper tools and
+abstractions, even the simplest tasks will turn into a dirty hacks
+fest, then either into a maintenance nightmare, or simply into
+abandonware. Providing an empowering framework so that you don't get
+stuck in such predicaments is metalua's whole purpose. The central
+concept is that programs prefer to manipulate code as trees, whereas
+most developers prefer ASCII sources, so both representations must be
+freely interchangeable. The make-or-break deal is then:
- To easily let users see sources as trees, as sources, or as
combination thereof, and switch representations seamlessly.
compiler-writing specialized languages (and which has nothing to do
with string regular expressions BTW), which lets you express
advanced tree analysis operations in a compact, readable and
- efficient way. If you regularly have to work with advanced data
- structures and you try it, you'll never go back.
+ efficient way. If you have to work with advanced data structures
+ and you try it, you'll never go back.
- The walker library allows you to perform transformations on big
portions of programs. It lets you easily express things like:
error statements", "rename all local variables and their instances
into unique fresh names", "list the variables which escape this
chunk's scope", "insert a type-checking instruction into every
- assignments to variable X", etc. You can't write many non-trivial
- macros without needing to do some of those global code
- transformations.
+ assignments to variable X", etc. Most of non-trivial macros will
+ requir some of those global code transformations, if you really want
+ them to behave correctly.
- Macro hygiene, although not perfect yet in metalua, is required if
you want to make macro writing reasonably usable (and contrary to a
- The compiler/interpreter front-end is completely rewritten. The new
- program, aptly named 'metalua', supports proper passing of arguments
- to programs, and is generally speaking much more user friendly than
- the mlc from the previous version.
+ frontend program, aptly named 'metalua', supports proper passing of
+ arguments to programs, and is generally speaking much more user
+ friendly than the mlc from the previous version.
- Metalua source libraries are looked for in environmemt variable
native Lua compiler.
By convention, metalua source files should have extension .mlua. By
- default, bytecode and plain lua files are preferred to metalua
- sources, which lets you easily precompile your libraries.
+ default, bytecode and plain lua files have higher precedence than
+ metalua sources, which lets you easily precompile your libraries.
- Compilation of files are separated in different Lua Rings: this
include:
- Lua Rings and Pluto, duct-taped together into Springs, an improved
Rings that lets states exchange arbitrary data instead of just
- scalars and strings.
+ scalars and strings. Since Pluto requires a (minor) patch to the
+ VM, it can be disabled.
- Lua bits for bytecode dumping.
- As always, very large amounts of code borrowed from Yueliang.
- As a commodity, I've also packaged Lua sources in.
+- Extensions to Lua standard libraries: many more features in table
+ and the baselib, a couple of string features, and a package system
+ which correctly handles metalua source files.
+
+
- Builds on Linux, OSX, Microsoft Visual Studio. Might build on mingw
- (not tested recently). It's easily ported to all systems with a full
- support for lua.
-
- The MS-windows building is hackish: it's driven by a batch script,
- and Pluto can't compile as a win32 DLL, so it's linked in the Lua
- VM. If you want to run your own VM, either link pluto in statically,
- or disabled separate compilation by setting environment variable
- LUA_MFAST at true. In the later case, expect puzzling behaviors when
- you load several sources containing compile-time code (==>
- precompile everything).
-
- Notice that bits of the compiler itself are now written in metalua,
+ (not tested recently, patches welcome). It's easily ported to all
+ systems with a full support for lua, and if possible dynamic
+ libraries.
+
+ The MS-windows building is based on a dirty .bat script, because
+ that's pretty much the only thing you're sure to find on a win32
+ computer. It uses Microsoft Visual Studio as a compiler (tested with
+ VC++ 6).
+
+ Notice that parts of the compiler itself are now written in metalua,
which means that its building now goes through a bootstrapping
stage.
-- Structural pattern matching:
+- Structural pattern matching improvements:
- now also handles string regular expressions: 'someregexp'/pattern
will match if the tested term is a string accepted by the regexp,
and on success, the list of captures done by the regexp is matched
- Overhaul of the code walking library:
- the API has been simplified: the fancy predicates proved more
- cumbersome to use than a bit pattern matching in the visitors
+ cumbersome to use than a bit of pattern matching in the visitors.
- binding identifiers are handled as a distinct AST class
- walk.id is scope-aware, handles free and bound variables in a
sensible way.
macro system with v0.4, but what exists remains a work in
progress. Lua is a Lisp-1, which means unhygienic macros are very
dangerous, and hygiene a la Scheme pretty much limits macro writing
- to a term rewriting subset of the language, which is crippling to
- use.
-
- Note: inside hygiene, i.e. local variables created by the macro
- which might capture user's variable instances, is trivial to address
- by alpha conversion. The trickier part is outside hygiene, when
- user's binders capture globals required by the macro-generated
- code. That's the cause of pretty puzzling and hard to find bugs. And
- the *really* tricky part, which is still unsolved in metalua, is
- when you have several levels of nesting between user code and macro
- code. For now this case has to be hygienized by hand.
-
- Note 2: Converge has a pretty powerful approach of hygienic macros
- in a Lisp-1 language; for long and boringly technical reasons, I
- don't think its approch would be the best suited to metalua.
+ to a term rewriting subset of the language, which would be crippling
+ to use.
+
+ Note: inside hygiene, i.e. preventing macro code from capturing
+ variables in user code, is trivial to address through alpha
+ conversion, it's not the issue. The trickier part is outside
+ hygiene, when user's binders capture globals required by the
+ macro-generated code. That's the cause of pretty puzzling and hard
+ to find bugs. And the *really* tricky part, which is still an open
+ problem in metalua, is when you have several levels of nesting
+ between user code and macro code. For now this case has to be
+ hygienized by hand.
+
+ Note 2: Converge has a pretty powerful approach to hygienic macros
+ in a Lisp-1 language; for reasons that would be too long to expose
+ here, I don't think its approch would be the best suited to metalua.
+ But I might well be proved wrong eventually.
Note 3: Redittors must have read that Paul Graham has released Arc,
which is also a Lisp-1 with Common Lisp style macros; I expect this
hygiene, this will probably be the closest we can go to "macros for
dummies" without creating an unmaintainable mess generator.
+ Besides, it's consistent with my official position that focusing on
+ superficial syntax issues is counter-productive most of the time :)
+
- Lexers can be switched on the fly. This lets you change the set of
- keywords temporarily, with the new gg.with_lexer() functor, or
- handle radically different syntaxes in a single file (think
+ keywords temporarily, with the new gg.with_lexer() combinator. You
+ can also handle radically different syntaxes in a single file (think
multiple-languages systems such as LuaTeX, or programs+goo as PHP).
-- Incorporation of the bugs listed on the mailing list and the blog.
+- Incorporation of the bug fixes reported to the mailing list and on
+ the blog.
- New samples and extensions, in various states of completion:
- - lists by comprehension, a la python/haskell. It includes lists
+ * lists by comprehension, a la python/haskell. It includes lists
chunking, e.g. mylist[1 ... 3, 5 ... 7]
- - anaphoric macros for 'if' and 'while' statements: with this
+ * anaphoric macros for 'if' and 'while' statements: with this
extension, the condition of the 'if'/'while' is bound to variable
'it' in the body; it lets you write things like:
- while file:read '*l' do print(it) end.
+
+ > while file:read '*l' do print(it) end.
+
No runtime overhead when 'it' isn't used in the body. An anaphoric
variable should also be made accessible for functions, to let
easily write anonymous recursive functions.
- - continue statement, logging facility, ternary "?:" choice operator
-
- - Try ... catch ... finally extension.
+ * Try ... catch ... finally extension. Syntax is less than ideal,
+ but the proper way to fix that is to refactor the match extension
+ to improve code reuse. There would be many other greate ways to
+ leverage a refactored match extension, e.g. destructuring binds or
+ multiple dispatch methods. To be done in the next version.
- - with ... do extension: it uses try/finally to make sure that
+ * with ... do extension: it uses try/finally to make sure that
resources will be properly closed. The only constraint on
- resources is that they have to sport a :close() releasing method.
+ resources is that they have to support a :close() releasing method.
For instance, he following code guarantees that file1 and file2
will be closed, even if a return or an error occurs in the body.
- with file1, file2 = io.open "f1.txt", io.open "f2.txt" do
- contents = file1:read'*a' .. file2:read ;*a'
- end
+ > with file1, file2 = io.open "f1.txt", io.open "f2.txt" do
+ > contents = file1:read'*a' .. file2:read ;*a'
+ > end
+
+ * continue statement, logging facilities, ternary "?:" choice
+ operator, assignments as expressions, and a couple of similarly
+ tiny syntax sugar extensions.
+
+
+You might expect in next versions
+=================================
+The next versions of metalua will provide some of the following
+improvements, in no particular order: better error reporting,
+especially at runtime (there's a patch I've been too lazy to test
+yet), support for 64 bits CPUs, better support for macro hygiene, more
+samples and extensions, an adequate test suite, refactored libraries.
Credits
=======
I'd like to thank the people who wrote the open source code which
makes metalua run: the Lua team, the authors of Yueliang, Pluto, Lua
-Rings, Bitlib; the people whose bug reports, patches and insightful
-discussions dramatically improved the global design, including John
-Belmonte, Viacheslav Egorov, David Manura, Eric Raible, Laurence
-Tratt...
+Rings, Bitlib; and the people whose bug reports, patches and
+insightful discussions dramatically improved the global design,
+including John Belmonte, Viacheslav Egorov, David Manura, Olivier
+Gournet, Eric Raible, Laurence Tratt...
\paragraph{Unary Operators}
Unary operations are similar to binary operators, except that they
-only take the AST of one subexression. The following table associates
+only take the AST of one subexression. The following table associates
a Lua unary operator to its AST:
\begin{center}
\bf Op. & \bf AST \\
\hline\hline %%%%%%%%%%%%%%
- \verb+-+ & \verb+"sub"+ &
+ \verb+-+ & \verb+"unm"+ &
\verb+#+ & \verb+"len"+ &
\verb+not+ & \verb+"not"+ \\
\hline %%%%%%%%%%%%%%%%%%%%
\subparagraph{Examples}
\begin{itemize}
-\item \verb|-x| is represented as \verb|`Op{ `Sub, `Id "x" }|;
+\item \verb|-x| is represented as \verb|`Op{ 'unm', `Id "x" }|;
\item \verb|-(1+2)| is represented as:\\
- \verb|`Op{ `Sub, `Op{ `Add, `Number 1, `Number 2 } }|
+ \verb|`Op{ 'unm', `Op{ 'add', `Number 1, `Number 2 } }|
\item \verb|#x| is represented as
- \verb|`Op{ `Len, `Id "x" }|
+ \verb|`Op{ 'len', `Id "x" }|
\end{itemize}
\paragraph{Indexed access}
-PLATFORMS = macosx mingw
+PLATFORMS = macosx mingw linux
include config
.PHONY: libraries install clean
%.$(LIBEXT): %.$(OBJEXT)
- $(MKLIB) -o $@ $+
+ $(MKLIB) -o $@ $+ $(LDFLAGS)
LUA_BINARIES = $(COMPILE).exe $(RUN).exe metalua.dll
endif
+ifeq ($(PLATFORM), linux)
+ LIBEXT = so
+ MKLIB = gcc -shared
+ LDFLAGS = -L../$(LUA_VM_DIR) -llua
+ LUA_BINARIES = $(COMPILE) $(RUN)
+endif
+
#########################################################
# Environment variables settign paths
+++ /dev/null
---------------------------------------------------------------------------------
--- Tables that automatically generates subtables upon access.
---------------------------------------------------------------------------------
--- Borrowed from Lua-users, the Lua wiki.
--- (c) Thomas Wrensch & Rici Lake
---------------------------------------------------------------------------------
---
--- For instance, autotable().a.b.c = 42 is legal.
--- TODO: add proper pairs() and ipairs() iterators.
---------------------------------------------------------------------------------
-
-local meta, auto, assign
-
-function auto(tab, key)
- return setmetatable({}, {
- __index = auto,
- __newindex = assign,
- parent = tab,
- key = key
- })
-end
-
---------------------------------------------------------------------------------
--- The if statement below prevents the table from being created if the
--- value assigned is nil. This is, I think, technically correct but it
--- might be desirable to use assignment to nil to force a table into
--- existence.
---------------------------------------------------------------------------------
-function assign(tab, key, val)
- -- if val ~= nil then
- local oldmt = getmetatable(tab)
- oldmt.parent[oldmt.key] = tab
- setmetatable(tab, meta)
- tab[key] = val
- -- end
-end
-
-meta = {__index = auto}
-
-function autotable()
- return setmetatable({}, meta)
-end
-
-
-- free_vars is an old_name -> new_name dictionary computed from alpha:
-- self.alpha is not an efficient representation for searching.
if not alpha then alpha = { }; self.alpha = alpha end
+ -- FIXME: alpha should only be overridden when there actually are some
+ -- globals renamed.
if #alpha==0 then alpha <- `Local{ { }, { } } end
local new, old = unpack(alpha)
local free_vars = { }
+++ /dev/null
-Machine = { }
-Machine.__index = Machine
-
---------------------------------------------------------------------------------
--- Default fields for new machine instances
---------------------------------------------------------------------------------
-Machine.template = {
- fenv = { },
- state = 'init',
- states = { },
- action_code = { } }
-
---------------------------------------------------------------------------------
--- Events must be one of these. If not, cause a runtime error.
---------------------------------------------------------------------------------
-Machine.legal_events = table.transpose{ 'enter', 'update', 'exit' }
-
---------------------------------------------------------------------------------
--- Create a new machine isntance. [x] can be nil, or a table whose fields will
--- override the defaults provided by Machine.template.
---------------------------------------------------------------------------------
-function Machine:new(x)
- local t = table.deep_copy (self.template)
- if x then table.override(t, x) end
- return setmetatable(t, self)
-end
-
---------------------------------------------------------------------------------
--- [process_msg] expects the machine [self] to be waiting for a message.
--- It gets such a message, does the appropriate state transition and
--- causes the appropriate event-triggered actions to be run.
--- Called by Machine:message() a.k.a. Machine:__call().
---------------------------------------------------------------------------------
-local function process_msg (self, msg)
-
- -- Arg checking.
- assert (type(msg)=='string', "Machine messages must be strings")
- -- When there's a queue, new msgs are queued rather than executed directly.
- self.msg_queue = { }
-
- ---------------------------------------------------------
- -- Determine the source and destination states.
- ---------------------------------------------------------
- local src_state = self.state
- local dst_state = self.states[src_state].transitions[msg]
- if not dst_state then
- error("Machine has no transition from state "..src_state..
- " triggered by message "..msg)
- end
-
- ---------------------------------------------------------
- -- Run the appropriate actions.
- ---------------------------------------------------------
- if src_state==dst_state then
- local update_actions = self.states[src_state].actions.update
- for idx in ivalues(update_actions) do self.action_code[idx]() end
- else
- local exit_actions = self.states[src_state].actions.exit
- local enter_actions = self.states[dst_state].actions.enter
- for idx in ivalues(exit_actions) do self.action_code[idx]() end
- for idx in ivalues(enter_actions) do self.action_code[idx]() end
- end
-
- ---------------------------------------------------------
- -- Are there more messages queued? if so, process the next
- -- one; if not, allow to process more messages directly
- ---------------------------------------------------------
- local next_msg = table.remove (self.msg_queue)
- if next_msg then return process_msg (self, next_msg) -- tail call
- else self.msg_queue = nil end -- ready for direct msg accept.
-end
-
---------------------------------------------------------------------------------
--- Send or queue a message to the machine, depending on whether it is already
--- processing a message.
--- Also registered as the machine's __call metamethod.
---------------------------------------------------------------------------------
-function Machine:message (msg)
- if self.msg_queue then table.insert(self.msg_queue, msg)
- else process_msg (self, msg) end
-end
-Machine.__call = Machine.message
-
---------------------------------------------------------------------------------
--- add a new code to execute whenever one of the events is triggered from one
--- of the states. Lists of events and states can be replaced by a raw string
--- when they would have been singletions: ":add_action({x}, {y}, z)" is the
--- same as ":add_action(x, y, z)".
---------------------------------------------------------------------------------
-function Machine:add_action(states, events, code)
- if type(states)~='table' then states={states} end
- if type(events)~='table' then states={events} end
- table.insert(self.action_code, code)
- local action_idx = #self.action_code
- for state in ivalues (states) do
- for event in ivalues (events) do
- if not self.legal_events[event] then
- error ("Illegal event "..table.tostring(event))
- elseif not self.states[state] then
- error ("No state "..table.tostring(state).." in this machine")
- else
- table.insert(self.states[state].actions, action_idx)
- end
- end
- end
-end
-
---------------------------------------------------------------------------------
--- Add a new state. Causes a warning, but no error if the state already exists.
---------------------------------------------------------------------------------
-function Machine:add_state(state_name)
- assert (type(state_name)=='string', "Machine states must be strings")
- if self.states[state_name] then
- printf ("*** Warning: state %q already exists in machine. ***", state_name)
- else
- self.states[state_name] = {
- actions = { enter = { }, update = { }, exit = { } },
- transitions = { } }
- end
-end
-
---------------------------------------------------------------------------------
--- Add a new transition, from state src_state_name to dst_state_name, triggered
--- by msg. Cause a warning, but no error if there's already a transition from
--- this source state on this message, and override it.
---------------------------------------------------------------------------------
-function Machine:add_transition(src_state_name, msg, dst_state_name)
- -- Check args and states existence.
- assert (type(src_state_name)=='string', "Machine states must be strings")
- assert (type(dst_state_name)=='string', "Machine states must be strings")
- local src_state = self.states[src_state_name]
- local dst_state = self.states[dst_state_name]
- if not src_state then error("No state "..src_state.." in machine") end
- if not dst_state then error("No state "..dst_state.." in machine") end
-
- if src_state.transitions[msg] then
- printf ("*** Warning: overriding transition from state %s on msg %s. ***",
- src_state_name, msg)
- end
-
- src_state.transitions[msg] = dst_state_name
-end
-
-function Machine:restart()
- self.state = 'init'
- self.msg_queue = nil
-end
\ No newline at end of file
+++ /dev/null
--{ extension 'match' }
-
-require 'strict'
-
---[===[
--- The following concrete syntax: --
-machine
- state S1
- when enter, update do foo() end
- when enter do gnu() end
- when exit do bar() end
- on msg1 -> S1
- on msg2 -> S2
-end
-
--- Will be translated into the following config: --
-{ states = { S1 = { name = "S1",
- actions = { enter = { 1, 2 },
- update = { 1 },
- exit = { 2 } },
- transitions = { msg1 = "S1",
- msg2 = "S2" } } },
- action_code = { [1] = function() foo() end,
- [2] = function() gnu() end } }
- [3] = function() bar() end } }
---]===]
-
-----------------------------------------------------------------------
--- Definitions:
--- * an _event_ is relative to a state. It can be:
--- + 'enter', when we go in that state
--- + 'update', when we re-select the state we were already in.
--- + 'exit', when we leave the state
--- * an _action_ is a function, associated with a state and an event.
--- * a _message_ is a token sent to the state machine, which determines
--- the next state that will be selected.
--- * an _entry_ is a line of data, attached to a string, as produced
--- by the parser. This builder's main purpose is to turn such
--- entries into a proper machine config.
-----------------------------------------------------------------------
-local function machine_builder (x)
-
- --print "Building machine"
- local declared_states, used_states = { }, { }
- local actions = { }
- local last_action_idx = 0
-
- ---------------------------------------------------------
- -- First pass: gather the description of actions and
- -- transitions, sorted by affected states, in
- -- declared_states. state configs are indexed by name,
- -- and have fields:
- -- * name, their name;
- -- * code, an event -> function id hashtable
- -- * trans, a msg -> new state hashtable
- --
- -- Since actions can be bound to several events, they're
- -- associated with a unique index that will be used
- -- instead of the function itself, to prevent useless
- -- duplication. All those names are collected in the
- -- action_code hashtable, in the state's config.
- --
- -- This pass checks that there's no duplicate responses to
- -- messages or events.
- ---------------------------------------------------------
- for state in ivalues (x[1]) do
- local state_id, state_entries = unpack(state)
- local state_name = state_id[1]
- --printf(" Building state %s, pass #1", state_name)
- local state_cfg = declared_states[state_name]
- if not state_cfg then
- -- This state has no entry, create one.
- state_cfg = { name = state_name,
- transitions = { },
- actions = { enter = `Table{ },
- update = `Table{ },
- exit = `Table{ } } }
- declared_states[state_name] = state_cfg
- end
-
- for entry in ivalues (state_entries) do
- match entry with
- -----------------------------------------
- -- "when <id_list> do <block> end"
- -----------------------------------------
- | `When{ events, code } ->
-
- -- Set action_idx->action_code in cfg.action_code
- last_action_idx += 1
- local action_idx = last_action_idx
- actions [action_idx] = code
-
- -- Set ev_name->action_idx pairs in cfg[state].actions
- for ev_id in ivalues (events) do
- local ev_name = ev_id[1]
- --printf(" Parsing event %s", ev_name)
- local actions_list = state_cfg.actions[ev_name]
- if not actions_list then
- error ("Invalid event "..ev_name.." in state "..state_name)
- else
- table.insert(actions_list, `Number{action_idx})
- end
- end
- -----------------------------------------
- -- "on <msg_id> -> <state>"
- -----------------------------------------
- | `On{ messages, target_state } ->
-
- -- Set msg_name->ttarget_name pairs in cfg[state].transitions
- for msg_id in ivalues (messages) do
- local msg_name, target_name = msg_id[1], target_state[1]
- --printf(" Parsing transition %s -> %s", msg_name, target_name)
- if state_cfg.actions[msg_name] then
- error("Two transitions triggered by message "..msg_name..
- " in state "..state_name)
- end
- -- We'll later check that all target states exist.
- used_states[target_name] = { msg=msg_name, src_state=state_name }
- state_cfg.transitions[msg_name] = target_name
- end -- for each msg
- end -- match entry
- end -- for each entry
- end -- for each state
-
- ---------------------------------------------------------
- -- Check that every target state exists.
- ---------------------------------------------------------
- for a, b in pairs (used_states) do
- if not declared_states[a] then
- error ("State "..b.src_state.." tries to go to inexistant state "..a..
- " when receiving message "..b.msg)
- end
- end
-
- local states_ast = `Table{ }
-
- ---------------------------------------------------------
- -- 2nd pass: turn state configs into AST.
- ---------------------------------------------------------
- for state_cfg in values (declared_states) do
- --printf(" Pass #2 for state %s", state_cfg.name)
-
- -- Generate action name -> action code_idx hashtable.
- local actions_entry = `Table{ }
- for ev_name, action_indexes in pairs (state_cfg.actions) do
- table.insert (actions_entry,
- `Pair{ `String{ ev_name }, action_indexes })
- end
- -- Generate msg -> target state hastable.
- local transitions_entry = `Table{ }
- for msg_name, target_state in pairs (state_cfg.transitions) do
- table.insert (transitions_entry,
- `Pair{ `String{ msg_name }, `String{ target_state } })
- end
- -- Generate state config structure
- local state_ast =
- `Table{ `Pair{ `String 'actions', actions_entry },
- `Pair{ `String 'transitions', transitions_entry } }
- -- Register state structure.
- table.insert (states_ast, `Pair{ `String{ state_cfg.name }, state_ast })
- end
-
- ---------------------------------------------------------
- -- register the action name -> action code hashtable
- ---------------------------------------------------------
- local action_code_ast = `Table{ }
- for action_idx, action_code in pairs (actions) do
- table.insert (action_code_ast, `Pair{ `Number{ action_idx },
- `Function{ { }, action_code } })
- end
-
- local cfg_ast = +{ { action_code = -{action_code_ast},
- states = -{states_ast} } }
-
- return +{stat: Machine:new(-{cfg_ast})}
-end
-
-local machine_entry = gg.multisequence{
- { 'when', mlp.id_list, 'do', mlp.block, 'end', builder = 'When' },
- { 'on', mlp.id_list, '->', mlp.id, builder = 'On' },
- { 'do', mlp.block, 'end', builder =
- |x| `When{ { `Id 'enter', `Id 'update' }, x[1] } } }
-
-local machine_state = gg.sequence{
- 'state', mlp.id,
- gg.list{ machine_entry, terminators = { 'state', 'end' } } }
-
-local machine_content = gg.sequence { name = 'State machine constructor',
- gg.list{ machine_state, terminators = 'end' },
- builder = machine_builder }
-
-mlp.lexer:add{ 'machine', 'state', 'when', 'on', '->' }
-mlp.expr:add{ 'machine', machine_content, 'end', builder = |x|x[1] }
-mlp.stat:add{ 'machine', mlp.expr, machine_content, 'end',
- builder = |x| `Set{ {x[1]}, {x[2]} } }
-
-return +{ require 'extension.machine-runtime' }
\ No newline at end of file
\r
-- code handling the error catching process:\r
local catch_result do\r
- if #catch_cases>0 then\r
+ if catch_cases and #catch_cases>0 then\r
----------------------------------------------------------\r
-- Protect catch code against failures: they run in a pcall(), and\r
-- the result is kept in catch_* vars so that it can be used to\r
end
end
-return package
\ No newline at end of file
+return package
+++ /dev/null
-----------------------------------------------------------------------\r
--- Metalua samples: $Id$\r
---\r
--- Summary: AST --> source code converter.\r
---\r
-----------------------------------------------------------------------\r
---\r
--- Copyright (c) 2006, Fabien Fleutot <metalua@gmail.com>.\r
---\r
--- This software is released under the MIT Licence, see licence.txt\r
--- for details.\r
---\r
---------------------------------------------------------------------------------\r
---\r
--- This sample takes an AST and converts it into a source string.\r
--- Once correctly polished, this will be a valuable tool to debug\r
--- hairy macros that generate hard to read AST.\r
---\r
---------------------------------------------------------------------------------\r
---\r
--- TODO:\r
--- - handle precedence correctly. Currently, expressions requiring parentheses\r
--- might or might not work.\r
---\r
---------------------------------------------------------------------------------\r
-\r
--- Runtime warning\r
-print [[Warning: you are using srcdump, an AST -> source converter, which is not correct yet wrt e.g. precedence and parentheses. Expect nasty bugs.]]\r
--- Compile-time warning\r
--{ print [[Warning: you are using srcdump, an AST -> source converter, which is not correct yet wrt e.g. precedence and parentheses. Expect nasty bugs.]] }\r
-\r
--{ extension 'match' }\r
-\r
-local opnames = {\r
- __add = "+", __sub = "-",\r
- __mul = "*", __div = "/",\r
- __mod = "%", __pow = "^",\r
- __concat = "..", __eq = "==",\r
- __lt = "<", __le = "<=",\r
- __and = "and", __or = "or", \r
- __not = "not ", __len = "#" }\r
-\r
-local keywords = {\r
- "and", "break", "do", "else", "elseif",\r
- "end", "false", "for", "function", "if",\r
- "in", "local", "nil", "not", "or", "repeat",\r
- "return", "then", "true", "until", "while" }\r
-\r
-function dump (ast)\r
- local indent_offset = 0\r
- local accumulator = { }\r
- local function indent() indent_offset += 1 end\r
- local function dedent() indent_offset -= 1 end\r
- local function acc(...) table.insert(accumulator, string.format(...)) end\r
- local function newline() acc "\n"; acc (string.rep(" ", indent_offset)) end\r
- local is_ident = |x| string.strmatch(x, "^[%a_][%w_]*$") and not keywords[x]\r
-\r
- local parse_expr, parse_stat, parse_body, parse_funcdef, \r
- parse_id, parse_list, parse_block\r
-\r
- function parse_expr (ast)\r
- match ast with\r
- | `Nil -> acc "nil"\r
- | `Dots -> acc "..."\r
- | `True -> acc "true"\r
- | `False -> acc "false"\r
- | `Number{x} -> acc ("%i", x)\r
- | `String{x} -> acc ("%q", x)\r
- | `Id{ x } -> parse_id (x)\r
- | `Paren{ a } -> acc "("; parse_expr (a); acc ")"\r
-\r
- | `Op{ op, a, b } ->\r
- -- FIXME: handle precedence and parentheses around [a] and [b]\r
- parse_expr (a); acc (" %s ", opnames[op]); parse_expr (b)\r
-\r
- | `Op{ op, a } ->\r
- -- FIXME: handle precedence and parentheses around [a]\r
- acc ("%s", opnames[op]); parse_expr (a)\r
-\r
- | `Call{ f, `String{ _ } } | `Call{ f, `Table{ ... } } ->\r
- -- FIXME: handle precedence and parentheses around [f]\r
- parse_expr (f); acc " "; parse_expr (ast[2])\r
-\r
- | `Call{ f, ... } ->\r
- -- FIXME: handle precedence and parentheses around [f]\r
- parse_expr (f)\r
- acc " ("; parse_list (parse_expr, ast, ", ", 2); acc ")"\r
-\r
- | `Method{ f, name, ... } ->\r
- --CAVEAT: handle precedence and parentheses around [f]\r
- parse_expr (f)\r
- acc ":"; parse_id (name); acc " ("\r
- parse_list (parse_expr, ast, ", ", 3)\r
- acc ")"\r
-\r
- | `Function{ params, body } ->\r
- acc "function "; parse_funcdef (params, body)\r
-\r
- | `Index{ t, `String{ k } } if is_ident(k) ->\r
- --CAVEAT: handle precedence and parentheses around [t]\r
- parse_expr (t); acc (".%s", k)\r
-\r
- | `Index{ t, k } ->\r
- --CAVEAT: handle precedence and parentheses around [t]\r
- parse_expr (t); acc "["; parse_expr (k); acc "]"\r
- \r
- | `Table{ } -> acc "{ }"\r
-\r
- | `Table{ ... } ->\r
- acc "{"\r
- indent(); newline()\r
- for i = 1, #ast do\r
- match ast[i] with\r
- | `Pair{ `String{ k }, v } if is_ident (k)-> \r
- acc ("%s = ", k); parse_expr (v)\r
- | `Pair{ k, v } -> \r
- acc "["; parse_expr (k); acc "] = "; parse_expr (v)\r
- | v -> \r
- parse_expr (v)\r
- end\r
- if i < #ast then acc ";"; newline() end\r
- end\r
- acc " }"\r
- dedent()\r
-\r
- | `Stat{ b, e } ->\r
- acc "-{ `Stat{ +{ block:";\r
- parse_block (b)\r
- acc "}, +{ "\r
- parse_expr (e)\r
- acc " } } }"\r
-\r
- | _ -> error ("Can't sourcedump expression " .. ast.tag)\r
- end\r
- end\r
-\r
- function parse_funcdef (params, body)\r
- acc " ("; parse_list (parse_expr, params, ", "); acc ")"\r
- parse_block (body)\r
- acc "end"\r
- end\r
-\r
- function parse_block (ast)\r
- indent(); newline()\r
- for i = 1, #ast do\r
- parse_stat (ast[i])\r
- if i == #ast then dedent() end\r
- newline()\r
- end\r
- end\r
-\r
- function parse_id (ast)\r
- if type (ast) == "table" then ast = ast[1] end\r
- assert (type(ast) == "string", "Invalid identifier")\r
- if is_ident (ast) then acc ("%s", ast)\r
- else acc ("-{`Id %q}", ast) end -- FIXME: might require parens in some cases\r
- end\r
-\r
- function parse_list (parser, list, sep, start, finish)\r
- for i = start or 1, finish or #list do\r
- parser (list[i])\r
- if i<#list then acc ("%s", sep) end\r
- end\r
- end\r
-\r
- function parse_stat (ast)\r
- match ast with\r
- | `Do{ ... } -> acc "do"; parse_block (ast); acc "end"\r
-\r
- | `While{ cond, body } ->\r
- acc "while "; parse_expr (cond); acc " do"\r
- parse_block (body)\r
- acc "end"\r
-\r
- | `Repeat{ body, cond } ->\r
- acc "repeat"\r
- parse_block (body)\r
- acc "until "; parse_expr (cond)\r
-\r
- | `If{ ... } ->\r
- for i = 1, #ast-1, 2 do\r
- if i==1 then acc "if " else acc "elseif " end\r
- parse_expr (ast[i])\r
- acc " then"\r
- parse_block (ast[i+1])\r
- end\r
- if #ast % 2 == 1 then\r
- acc "else"\r
- parse_block (ast[#ast])\r
- end\r
- acc "end" \r
-\r
- | `Fornum{ `Id{ var }, a, b, c, body } ->\r
- acc "for "; parse_id (var); acc " = "\r
- parse_expr (a); acc ", "\r
- parse_expr (b); acc ", "\r
- parse_expr (c); acc " do"\r
- parse_block (body)\r
- acc "end"\r
-\r
- | `Fornum{ `Id{ var }, a, b, body } ->\r
- acc "for "; parse_id (var); acc " = "\r
- parse_expr (a); acc ", "\r
- parse_expr (b); acc " do"\r
- parse_block (body)\r
- acc "end"\r
-\r
- | `Forin{ varlist, vallist, body } ->\r
- acc "for "; parse_list (parse_id, varlist, ", "); acc " = "\r
- parse_list (parse_expr, vallist, ", ")\r
- acc " do"\r
- parse_block (body)\r
- acc "end"\r
-\r
- | `Local{ lhs, { } } ->\r
- acc "local "\r
- parse_list (parse_id, lhs, ", ")\r
-\r
- | `Local{ lhs, rhs } ->\r
- acc "local "\r
- parse_list (parse_id, lhs, ", ")\r
- acc " = "\r
- parse_list (parse_expr, rhs, ", ")\r
-\r
- | `Localrec{ { `Id{ name } }, { `Function{ params, body } } } ->\r
- acc "local function "; parse_id (name); parse_funcdef (params, body)\r
-\r
- | `Set{ { `Index{ obj, `String{ name } } }, \r
- { `Function{ { `Id "self", ... }, body } } } ->\r
- local params = ast[2][1][1]\r
- acc "function "\r
- parse_expr (obj); acc (":%s ", name); parse_funcdef (params, body)\r
- \r
- | `Set{ { lhs }, { `Function{ params, body } } } ->\r
- acc "function "; parse_expr (lhs); parse_funcdef (params, body)\r
-\r
- | `Set{ lhs, rhs } -> \r
- parse_list (parse_expr, lhs, ", ")\r
- acc " = "\r
- parse_list (parse_expr, rhs, ", ")\r
-\r
- | `Return{ ... } -> acc "return "; parse_list (parse_expr, ast, ", ")\r
- | `Goto{ x } -> acc "-{ `Goto{ +{ "; parse_expr (x); acc " } } }" \r
- | `Label{ x } -> acc "-{ `Label{ +{"; parse_expr (x); acc " } } }" \r
-\r
- | { ... } if not ast.tag -> \r
- for i = 1, #ast do\r
- parse_stat (ast[i])\r
- if i<#ast then newline() end\r
- end\r
- \r
- | `Break -> acc "break"\r
- | `Call{ ... } -> parse_expr (ast)\r
- | `Invoke{ ... } -> parse_expr (ast)\r
- | _ -> error ("Can't sourcedump statement " .. ast.tag)\r
- end\r
- end\r
- \r
- local expr_tags = { \r
- Nil = 1, Dots = 1, True = 1, False = 1, Number = 1, String = 1, \r
- Function = 1, Table = 1, Call = 1, Method = 1, Id = 1, Index = 1,\r
- Op = 1 }\r
-\r
- if expr_tags [ast.tag] then parse_expr (ast) else parse_stat (ast) end\r
- return table.concat (accumulator)\r
-end
\ No newline at end of file
** (versions 3.2 and later) mark them as "hidden" to optimize access
** when Lua is compiled as a shared library.
*/
+#if 1 /* Metalua: export internal symbols needed by Pluto. */
+# define LUAI_FUNC LUA_API
+# define LUAI_DATA LUA_API
+#else /* Metalua: Original version, disabled */
#if defined(luaall_c)
#define LUAI_FUNC static
#define LUAI_DATA /* empty */
#define LUAI_FUNC extern
#define LUAI_DATA extern
#endif
+#endif
+++ /dev/null
--{ extension 'machine' }
-
-machine M
- state init
- on MSG1, MSG2 -> S1
- state S1
- when enter do
- print "do whatever you want in this Lua block"
- print "it will be evaluated every time the machine enters state S1"
- end
- on SOME_MSG -> S2 -- When the machine is in state S1 and receives msg
- -- SOME_MSG, jump to state S2.
- state S2
- do
- print "there's no explicit 'when' before this action block"
- print "It's run when entering and updating the state by default"
- end
- on SOME_OTHER_MSG, OR_ANOTHER_MSG -> S1
- end
-
-table.print(M, 60)
\ No newline at end of file
require 'metalua.compiler'
---assert (+{foo}.quote, "no q!?")
-
--- Usage samples:
-
TEST_CASES = {
{ "everything should be renamed",
+{ block:
print(i)
-{!`Call{`Id 'print', `String 'hi' } }
- -- Bugger. I don't detect any non-quoted element , so
- -- the -{+{ }} thingie won't protect from capture.
-{!+{print 'hi'}} },
{ } },
print (x, y) },
{ alpha = +{stat: local RENAMED_PRINT = print},
keep = {y = true} } } }
-
--- { +{ block: --
--- },
--- { } },
for case in ivalues(TEST_CASES) do
local comment, ast, cfg = unpack(case)
require 'dollar'
- local H = H:new{alpha={ }}
+ local H = H:new()
print("initial H.alpha", H.alpha)
------------------------------------------------------------
assert (x == 3)
+print "It seems to work..."
\ No newline at end of file
+-- This is a simple and somewhat stupid example of how to switch
+-- lexers dynamically. Behind a V, X and Y are the only reserved
+-- keywords. In normal conditions, X and Y aren't keywords and can be
+-- used as variables.
+
-{ block:
require 'lexer'
local my_lexer = lexer.lexer:clone() -- no keywords
disp = |msg,ast| printf("\n%s:\n%s", msg, table.tostring(ast, 80, 'nohash'))
disp('initial term', ast)
-require'autotable'
-
do -- Make globals explicit:
local ast = table.deep_copy(ast)
local cfg = { id = { } }
disp('Local alpha-renamed', ast)
end
-
---[=[
-
-
-local varlist_mt = { __index = function (self, key)
- local x={ }; self[key] = x; return x
- end }
-local bound_vars = setmetatable({ }, varlist_mt)
-local free_vars = setmetatable({ }, varlist_mt)
-
-cfg = {
- id = {
- bound = |x, binder| table.insert(bound_vars[binder], x),
- free = |x| table.insert(free_vars[x[1]], x) } }
-
-walk_id.block(cfg, x)
-
--- Alpha convert bound variables
-for binder, occurences in pairs(bound_vars) do
- local new_name = "_LOCAL_"..occurences[1][1]
- local cfg = { binder = function (x) x[1]=new_name; return 'break' end }
- walk.guess(cfg, binder)
- for occ in ivalues(occurences) do occ[1] = new_name end
-end
-
--- Alpha convert free variables
-for name, occurences in pairs(free_vars) do
- local new_name = "_GLOBAL_"..occurences[1][1]
- for occ in ivalues(occurences) do occ[1] = new_name end
-end
---]=]
-
---print "\n\n TERM AFTER PARSING:"
---table.print(x,60,'nohash')
-
-set MLUALIB_TARGET=d:\fft\tmp\mlualib\r
-set MSVCDIR=c:\devtools\msvc\vc98\bin\r
+set MLUALIB_TARGET=c:\tmp\mlualib\r
+set MSVCDIR=c:\msvc\vc98\bin\r
md %MLUALIB_TARGET%\r
\r
:lua\r
@REM Code taken straight from Lua's etc/luavs.bat\r
cd lua\r
%MSVCDIR%\cl /nologo /MD /O2 /W3 /c /D_CRT_SECURE_NO_DEPRECATE /D_CRT_NONSTDC_NO_DEPRECATE /DLUA_BUILD_AS_DLL l*.c\r
-%MSVCDIR%\cl /nologo /MD /O2 /W3 /c /D_CRT_SECURE_NO_DEPRECATE /D_CRT_NONSTDC_NO_DEPRECATE /DLUA_BUILD_AS_DLL /DLUA_CORE /I . ../binlibs/pluto.c\r
del lua.obj luac.obj\r
-%MSVCDIR%\link /nologo /DLL /out:lua51.dll l*.obj pluto.obj\r
+%MSVCDIR%\link /nologo /DLL /out:lua51.dll l*.obj\r
%MSVCDIR%\cl /nologo /MD /O2 /W3 /c /D_CRT_SECURE_NO_DEPRECATE /D_CRT_NONSTDC_NO_DEPRECATE /DLUA_BUILD_AS_DLL lua.c\r
%MSVCDIR%\link /nologo /out:lua.exe lua.obj lua51.lib\r
%MSVCDIR%\cl /nologo /MD /O2 /W3 /c /D_CRT_SECURE_NO_DEPRECATE /D_CRT_NONSTDC_NO_DEPRECATE l*.c print.c\r
cd binlibs\r
%MSVCDIR%\cl /nologo /LD /DLUA_BUILD_AS_DLL /DLUA_LIB /DBUILTIN_CAST /I..\lua bit.c ..\lua\lua51.lib\r
%MSVCDIR%\cl /nologo /LD /DLUA_BUILD_AS_DLL /DLUA_LIB /I..\lua rings.c ..\lua\lua51.lib\r
+%MSVCDIR%\cl /nologo /LD /DLUA_BUILD_AS_DLL /DLUA_LIB /I..\lua pluto.c ..\lua\lua51.lib\r
xcopy /Y rings.dll %MLUALIB_TARGET%\r
-xcopy /Y bit.dll %MLUALIB_TARGET%\r
+xcopy /Y bit.dll %MLUALIB_TARGET%\r
+xcopy /Y pluto.dll %MLUALIB_TARGET%\r
cd ..\r
\r
:w32stub\r
-@REM Pluto has been compiled into lua, for DLL exportation reasons. Provide a stub\r
-cd win32\r
-%MSVCDIR%\cl /nologo /c /DLUA_BUILD_AS_DLL /I..\lua pluto_w32_stub.c\r
-%MSVCDIR%\link /nologo /DLL /out:pluto_w32_stub.dll pluto_w32_stub.obj ..\lua\lua51.lib\r
-xcopy /Y pluto_w32_stub.dll %MLUALIB_TARGET%\r
-xcopy /Y pluto.lua %MLUALIB_TARGET%\r
echo @set LUA_ROOT=%MLUALIB_TARGET%> %MLUALIB_TARGET%\metalua.bat\r
-type metalua.bat >> %MLUALIB_TARGET%\metalua.bat\r
-cd ..\r
+type win32\metalua.bat >> %MLUALIB_TARGET%\metalua.bat\r
\r
:setenv\r
@REM set Metalua environment\r
echo set LUA_PATH=?.luac;?.lua;%MLUALIB_TARGET%\?.luac;%MLUALIB_TARGET%\?.lua >> mlua_setenv.bat\r
echo set LUA_CPATH=?.dll;%MLUALIB_TARGET%\?.dll;%MLUALIB_TARGET%\?\linit.dll >> mlua_setenv.bat\r
echo set LUA_MPATH=?.mlua;%MLUALIB_TARGET%\?.mlua >> mlua_setenv.bat\r
-REM echo set PATH=%MLUALIB_TARGET%;%PATH% >> mlua_setenv.bat\r
+echo set PATH=%MLUALIB_TARGET%;%PATH% >> mlua_setenv.bat\r
CALL mlua_setenv.bat\r
\r
:compiler\r
projects / metalua.git / commitdiff