Lua: Improving deterministic resource cleanup

Design decisions

Ordered roughly by importance, here is the sequence of design decisions for a new resource cleanup (or finalization) mechanism in Lua.  If the Lua authors and finalization "usual suspects" in the community could agree on the answers to at least the first four items perhaps we could proceed in making this improvement to Lua.

  1. Should the Lua language provide better support for deterministic resource cleanup?
    Yes.  Many applications expose scarce resources to the scripting environment such as locks; handles for files, memory, and database transactions; etc.  Such resources need to be released promptly and deterministically after use, and despite any exceptional condition in the program.  Of course this can be accomplished with Lua as it is, but the resulting code is hard to follow1, highly error prone, and makes heavy use of protected calls-- limiting potential use of coroutines.
  2. Should this enhancement allow distinguishing of "exit" from "exit by error"?
    Yes.  A common use of the popular "try-catch" construct is simply to perform a side effect in the case of an error (for example, to roll back a transaction).  The error doesn't actually need to be "caught" in the sense that we don't need to transform it, suppress it, or even know any details about it.  Handling these situations with a unique cleanup idiom makes the program more clear and reduces programming errors, such as neglecting to re-raise an exception.
  3. Should the error object be provided to the cleanup code?
    Probably.  This seems harmless and would open the door for transformation or suppression of errors.  Normally this is accomplished with a try-catch construct, which Lua doesn't have.  If such a construct were separately added, indeed there is no reason to make the error object available in the cleanup case.
  4. Should cleanup code be injected by declarative syntax or meta mechanism?
    This being Lua, the answer is meta mechanism.  By way of contrast, the experimental finalize/guard enhancement presented on the Lua mailing list is an example of declarative syntax.  Cleanup code is added in special blocks opened with new "finalize" or "guard" keywords:
    function foo()
      local lock = aquire_lock()
      finalize lock:release() end
    The issue with a declarative cleanup block is that the resource user is compelled to write code.  Another example of declarative style is the popular try-catch-finally construct-- argued as poorly suited for resource finalization by the authors of the D programming language, "Exceptions in Lua" Gems article, and Lua finalize/guard patch.  Better would be a mechanism where the scope was made aware of certain objects, notifying them on exit-- in other words a user-defined hook for scope exits.  If such a mechanism existed, a declarative cleanup style could easily be implemented on top of it.  An example of this is the scope manager pattern from the "Exceptions in Lua" Gems article.
  5. Should the scoped object be designated by...
    1. a "scoped" variable class?
      That is, a keyword "scoped" which acts exactly as "local", except that the value is noted for signaling on scope exit or (perhaps, with additional implementation difficulty) when the variable is reassigned.  If there are several of these within a block level, they should probably be considered as sequentially nested.  That is, if we have "scoped a; scoped b", b would be cleaned up first, and any error in the cleanup code would be propagated to a's cleanup.
    2. metamethod magic?
      Similar to above but doesn't require a new keyword.  For example, add a new metamethod called "__exit".  Upon assignment of a local variable, the value is checked for this metamethod.  If it exists, the value is noted for signaling on exit.  The disadvantage here is that neither the compiler nor person reading the code can know they are dealing with scoped objects.  Also seems intrusive to a critical code path, namely locals assignment.
    3. a "with" block?
      This would be a new type of do-end block which essentially means "with a certain resource, do something, and then free the resource".  Python has this, and Metalua offers it as a language extension.
      • should it take the form "with VARLIST = EXP" or "with EXP [as VARLIST]"?
        The latter naturally emphasizes the importance of the scoped object, treating the name as secondary and optional.
    It should be noted there exist many more variants on the above three themes.  Overall, the "with" block seems preferable since it is explicit and provides a syntax which emphasizes the scoped value over any variable which it happens to be assigned to-- and in fact allows eliding of the variable altogether, which is useful in many common cases such as holding a lock.
  6. Should scope exit be signaled by callable interface, fixed method name, or new metamethod?
    This applies to "scoped" var and "with" block solutions-- various trade-offs here.  Callable allows simple function variables and lambdas to be provides as the scoped EXP.  However this prevents potential uses of the callable interface within the block.  A fixed method name (Metalua's approach) removes this restriction while avoiding the need to deal with metatables.  Using a new metamethod, say __exit, is the most robust solution but can be tedious in simple cases.
  7. Should there be a scope entrance hook?
    No. This can be effected by requiring a call to instantiate the resource or manager, and following the convention of only making this call from the scoped EXP.

  1. Challenge: implement Alex Mania's appendud example function in plain Lua while keeping the code easy to follow.  See <>.