RCR 234: Uniform meta-access while parsing Ruby code (replaces RCR 228)

Ruby exposes self within a class body, and also calls certain methods (e.g. Module#included) to signal certain parsing events (e.g. including of a module). These are powerful extensibility feature, allowing new methods to be written at the meta-level (e.g. Class) and easily invoked at many places within Ruby code. This RCR provides both extensibility mechanisms uniformaly throughout the source code, with access to many other syntactic elements (methods, parameters, constants, ...).

Facilities such as attr rely on methods on Class being invoked within the text of a class fdefinition with self = current_class. I can write new methods on Class and explicitly invoke them directly from anywhere within a class that self is bound.

Similarly, methods such as Module#included and Class#inherited are always invoked by the Ruby intepreter on certain events like module inclusion or class inheritance. I can override these methods for my own module or class and they are automatically invoked when those events take place with my module or class.

However, these two facilities are limited. I cannot define new methods on Method (or UnboundMethod, or whatever the underlying meta-level class will be called) and explicitly invoke them (similar to today's common attr invocation) at appropriate points (if any) in a method definition. I cannot write new methods on Parameter ... etc.

Similarly, I cannot override methods on Method, Parameter, etc. and have them automatically invoked by Ruby at key sentinel points in the code e.g. Method#started, ParamList#finished

This makes it more awkward than necessary to extend the meta-level e.g. I may want to create and attach information to explicit "Attribute" and "Method" objects to represent various meta-data information

This facility could be extremely useful if more uniformly available. Classes representing the entire Ruby Abstract Syntax Tree should be published (Method, Parameter, Constant, ...), self should be bound to instances of these AST classes during parsing, and a rich set of events should be defined that call methods on those AST objects (Class#started, Class#finished, Module#included, Class#inherited, ParamList#started, ParamList#finished, etc.) just as SAX parsers raise events when procesing an XML stream).

• Expose classes for as many AST elements as possible: Method, Parameter, Constant, perhaps even more fine grained such as LocalVariable, InstanceVaeiable, Block, Expression.
• Define a full set of standard methods on those classes, similar to today's Module#included or Class#inherited. Examples would be Method#wrapped (for the 2.0 proposed method wrapping facility), Param#started, etc.
• Invoke those methods (with empty default implementations) at appropriate points / events in the source code.
• Allow these AST classes to be extended with new methods (of course).
• Ensure that self is bound to the most current AST object in as many places as possible in the grammar.

The examples below are meant only to illustrate the idea. I do not have detailed enough knowledge of the Ruby grammar to know which points in the code would be reasonable choices for definition of new methods or explicit invocation of methods, or if some syntax change may be involved . I have annotatated the code with [1][2]... as points in the code that I cross-reference subsequently.

class Method [1]
 def started [2]; p "New Method"; end
 def explicit [3]; p "Method#explicit"; end
end

class [4] A [5]
 def [6] foo [7] explicit [8] (a [9] ...)[10]
   # body
 end [11]  def self.wrapped [12] (...) # used like self.included in a class
         end [13]
 [14]
end [15]

> Explanation of above code refers to [1], [2], etc.

• [1] Extending the pre-defined class Method
• [2] Re-defining a standard method invoked when a method definition is encountered
• [3] Defining a new method that will have to be explicit invoked
• [4] self = the new Class here
• [5] Predefined method Class#named invoked
• [6] self = the new Method here
• ; Method#started invoked
• [7] Predefined method Method#named invoked here
• [8] Method#explicit invoked explicitly here
• [9] self = the new Param here
• ; Param#started invoked
• [10] self = the ParamList here; ParamList#finished invoked
• [11] self = the new Method object here; hence ...
• [12] self.wrapped defines singleton version of predefined Method#wrapped
• [13] self is still bound to the "self.wrapped" method object
• [14] self = the Class object here
• [15] self still = the Class object here

With this scheme:

• Many places that refer to Method could better refer to (an improved) UnboundMethod
• New meta-level methods and meta-data can be defined and used easily
• The AST is available incrementally while parsing
• New level of extending Ruby at the meta-level for typing, documentation, code generation, analysis, method combination, aspects, etc.
• The AST objects exposed should also have published methods to traverse the structure e.g. from Param to Method to Class.
• Some of the finer-grained methods (e.g.on Param, or on ParamList) can be "lifted" to methods on their containing or parent objects (e.g. Method or Class) by adding extra parameters. I prefer access to finer grained AST objects as it has clear advantages

If I knew more of how the Ruby grammar was being implemented I could probably suggest something. However, most of this seems to require a generalization of facilities currently available: self binding during parsing, and automatic invocation of pre-defined methods at key events during parsing.