ruby picture

RCR 325: Inherit Module Class Methods

Submitted by transami (Sat Oct 29 00:15:31 UTC 2005)

Abstract

Many programs are forced to manually deal with the lack of Module class method inheritance. Common hacks to work around looks like this:

  module SomeModule
    module ClassMethods
      ...
    end
    def self.append_features(base)
      super
      base.extend(ClassMethods)
    end
    extend ClassMethods
    ...
  end

The inheritance of module class methods would benefit many well know Ruby applications like Rails and Nitro, while having negligable impact on programs that have no use for it.

Problem

Module class methods do not inherit through the class heirarchy.

  module M
    def self.t; "t"; end
  end
  class X
    include M
  end
  X.t #=> NoMethodError

This has led to all sorts of trickery. I myself may well have written the most clever way to overcome with Facets' #class_inherit --but it's still not 100% perfect becuase of interfernce from nasty code like ClassMethods hack above.

Some will bring up #extend, while it can be useful, general usage of extend is misconstrued b/c it thwarts encapsulation --encapsulation means a unified compontent, and as such should affect the class, include or extend, as it determines. This is evident by the very fact that so many do the above kind of hack. It's in Rails, it's in Og and Nitro, it's in much of my code too.

Proposal

Simply make Module's class/module methods inherited. And sooner the better! :)

Analysis

Do this, and many nasty hacks dissapear.

I don't think there is any major backward compatability issues either b/c any class methods that may already exist will override, so all we're getting is some extra methods in class space.

There's the minor possibility of interference between a module class method and a higher up class's class method, but that would be an exceedingly rare --even so isn't hard to fix in the unlikely case that it might occur.

Implementation

This wouldn't be the core implementation but I'll show you my work around b/c, like I said, it may be the ultimate attempt to overcome the pain.

 class Module
  alias_method :append_features_no_class_inherit, :append_features
  def append_features(base)
    result = append_features_no_class_inherit( base )
    class_inherit.each do |ext|
      base.extend( ext )
      unless base.is_a?(Class)
        bci = base.class_inherit
        bci << ext unless bci.include?(ext)
      end
    end
    result
  end
  def class_inherit(mod=nil,&blk)
    return (@_class_inherit||=[]) unless blk
    unless mod.is_a?(Module)
      mod = Module.new(&blk)
    end
    @_class_inherit ||= []
    @_class_inherit << mod
    extend( mod )
    mod
  end
 end
 class Class
  undef_method :class_inherit
 end

I'm sure a much more elegant means would be used in core.

ruby picture
Comments Current voting
I simply want to indicate that the 'append_features' example for including class methods need not be as cumbersome as the example demonstrates.

def self.included(base) base.extend(ClassMethods) end

Somewhat beside the point, 'append_features' is deprecated in favor of 'included'.


I think the point of things like the "included" hook is to preserve a flexible and scaleable language structure. "included" is not, in and of itself, a "nasty hack". It's actually a fairly modest reflective technique. If the particular use-case you're describing gets turned into a default, it would be very hard to make it *not* happen -- whereas, this way, it's quite easy to make it happen if you need it.

I also wouldn't describe the thing you're describing as "inheritance." I guess it's sort of coercing "include" into mimicking inheritance... but the whole thing about modules and incluce/extend is that it isn't the same mechanism as inheritance. It took me a while to reconcile myself to the idea that, given:

  class A
  end
  class B < A
  end
  def A.x
  end

B would respond to x. It makes sense in terms of the underlying model. I don't quite see how it would make sense with modules.

David Black


I simply want to indicate that the 'append_features' example for including class methods need not be as cumbersome as the example demonstrates.

  def self.included(base) base.extend(ClassMethods) end

I think you have missed the point of the implementation example. This simple "hack", which I do mention at the very beginning does not continue the inheritance chain.

T.


David,

  I think the point of things like the "included"
  hook is to preserve a flexible and scaleable
  language structure. "included" is not, in and
  of itself, a "nasty hack". It's actually a fairly
  modest reflective technique.

That's not what I am referening to. #included is a great hook, as is #inherited. The "hack" is getting module methods to participate in the class-level inheritance chain.

  If the particular use-case you're describing gets turned
  into a   default, it would be very hard to make it *not*
  happen -- whereas, this way, it's quite easy to make it
  happen if you need it.

We don't need it not to happen at all. If for some reason you must have a separated namespace, then make one.

  module MyModule
    module SeparateSpace
    end
  end

or

  module MyModule
  end
  module SeparateSpace
  end

You can't get any easier then that. But having to write a class_inherit hack as I have is *hard*, and still prone to breakage b/c, yes, it is a hack.

  I also wouldn't describe the thing you're describing as
  "inheritance." I guess it's sort of coercing "include" into
  mimicking inheritance... 

Huh? What are Classes' class methods doing then? The whole point is to have class-level inhertiance!

  but the whole thing about modules and incluce/extend is that it
  isn't the same mechanism as inheritance.

Sure it is. Modules participate in the instance level inheritance hierarchy in the same way classes do but via a proxy class. But this "proxying" is not complete having left out the class level.

  It took me a while to
  reconcile myself to the idea that, given:
    class A
    end
    class B < A
    end
    def A.x
    end
  B would respond to x. It makes sense in terms of the underlying
  model. I don't quite see how it would make sense with modules.

You had to reconcile your mind to that? I would suggest you not take such a theoretical viewpoint. Think of the pragmatics. If we couldn't do that how would we inherit DSLs, including #attr methods? This is a powerful feature, and just as it has been of great value with classes, so it would too for modules --which is more than evident by the fact that it is already happening in numerous projects (via the "hacks").

T.


Yes, I had to reconcile my mind to the idea that if you define a singleton method on one object, another object can call it. (I don't see what this has to do with attr_* methods.)

David


> (I don't see what this has to do with attr_* methods.)

We are only availed b/c Module is itself a class, not a module. But consider:

  class ActiveUpcase
    def self.attr_upcase(sym)
      class_eval %{
        def #{sym}
          @#{sym}.to_s.upcase
        end
        def #{sym}=(x)
          @#{sym} = x.to_s.upcase
        end
      }
    end
  end
  class MyClass < ActiveUpcase
    attr_upcase :bar
  end
  x = X.new
  x.bar = "hi"
  x.bar #=> "HI"

All well and good, right? But bump the class and give me a module instead --a common use case, and the whole thing falls.

  module ActiveUpcaseMixin
    def self.attr_upcase(sym)
      class_eval %{
        def #{sym}
          @#{sym}.to_s.upcase
        end
        def #{sym}=(x)
          @#{sym} = x.to_s.upcase
        end
      }
    end
  end
  class MyClass
    include ActiveUpcaseMixin
    attr_upcase :bar  # CRASH
  end

It's all about the DSLs. That's why the ways around the limitation are being used all over the place.

T.


You can't just post broken code, say that it's a "common use case", and use that as rationale for a language change :-)

In this particular case, it's actually easier to write a working one:

  module ActiveUpcaseMixin
     def attr_upcase
        ...
     end
   end
  class MyClass
    extend ActiveUpcaseMixin
    attr_upcase :x
  end

If you've got would-be class methods and instance methods in the same module, and you can't do everything you want with one include statement, it's probably a case where you need more modules (after all, the point is to be modular :-)

I also tend to think, just as a style or design matter, that very few class/module methods really should be called by two different class/modules. The whole point of defining SomeClass.meth is that meth is a facility or service that has some very specific connection with SomeClass. There's not much reason to favor the idea that it would make sense for subclasses of SomeClass to have the same facility -- and less so for a module.

It's interesting that if you do this:

class C; extend Math; end

C.sqrt and so on are private methods. You can't just use the C class object as if it were the Math module object -- and that's with extend. include is yet another level removed from the matter of giving class objects the functionality of module objects.

David


That's pedantic. I've clearly stated it is a common use case among some of the most popular application in Rubyland. Do you think I'm just making it up? Okay. Then I'll be pedantic too. Here's a .rb file grep on Rails and Nitro from my gems repostory, matching 'ClassMethods' and 'class_inherit':

actionmailer-1.1.3/lib/action_mailer/helpers.rb actionmailer-1.1.3/lib/action_mailer/adv_attr_accessor.rb actionpack-1.11.0/lib/action_controller/session_management.rb actionpack-1.11.0/lib/action_controller/caching.rb actionpack-1.11.0/lib/action_controller/helpers.rb actionpack-1.11.0/lib/action_controller/dependencies.rb actionpack-1.11.0/lib/action_controller/filters.rb actionpack-1.11.0/lib/action_controller/macros/in_place_editing.rb actionpack-1.11.0/lib/action_controller/macros/auto_complete.rb actionpack-1.11.0/lib/action_controller/rescue.rb actionpack-1.11.0/lib/action_controller/pagination.rb actionpack-1.11.0/lib/action_controller/benchmarking.rb actionpack-1.11.0/lib/action_controller/scaffolding.rb actionpack-1.11.0/lib/action_controller/layout.rb actionpack-1.11.0/lib/action_controller/verification.rb actionpack-1.11.0/lib/action_controller/upload_progress.rb actionwebservice-0.9.3/lib/action_web_service/container/action_controller_container.rb actionwebservice-0.9.3/lib/action_web_service/container/direct_container.rb actionwebservice-0.9.3/lib/action_web_service/container/delegated_container.rb actionwebservice-0.9.3/lib/action_web_service/api.rb actionwebservice-0.9.3/lib/action_web_service/invocation.rb actionwebservice-0.9.3/lib/action_web_service/scaffolding.rb actionwebservice-0.9.3/lib/action_web_service/dispatcher/action_controller_dispatcher.rb actionwebservice-0.9.3/lib/action_web_service/protocol/discovery.rb activerecord-1.13.0/lib/active_record/acts/list.rb activerecord-1.13.0/lib/active_record/acts/tree.rb activerecord-1.13.0/lib/active_record/acts/nested_set.rb activerecord-1.13.0/lib/active_record/validations.rb activerecord-1.13.0/lib/active_record/wrappings.rb activerecord-1.13.0/lib/active_record/observer.rb activerecord-1.13.0/lib/active_record/associations.rb activerecord-1.13.0/lib/active_record/deprecated_associations.rb activerecord-1.13.0/lib/active_record/callbacks.rb activerecord-1.13.0/lib/active_record/aggregations.rb activerecord-1.13.0/lib/active_record/reflection.rb activerecord-1.13.0/lib/active_record/transactions.rb activerecord-1.13.0/lib/active_record/wrappers/yaml_wrapper.rb activesupport-1.2.3/lib/active_support/core_ext/time/calculations.rb activesupport-1.2.3/lib/active_support/core_ext/load_error.rb facets-2005.10.15/lib/mega/crosscase.rb facets-2005.10.30/lib/mega/class_inherit.rb facets-2005.10.30/lib/facets.rb glue-0.24.0/lib/glue/validation.rb glue-0.24.0/lib/glue/aspects.rb glue-0.24.0/lib/glue/mailer/outgoing.rb glue-0.24.0/lib/glue/mailer/incoming.rb nitro-0.24.0/lib/nitro/scaffold.rb nitro-0.24.0/lib/nitro/caching/actions.rb nitro-0.24.0/lib/nitro/caching/output.rb nitro-0.24.0/lib/nitro/caching/invalidation.rb og-0.24.0/lib/og/relation.rb og-0.24.0/lib/og/entity.rb rails-0.14.3/lib/rails_generator/options.rb rails-0.14.3/lib/rails_generator/lookup.rb actionmailer-1.1.3/lib/action_mailer/helpers.rb actionpack-1.11.0/lib/action_controller/helpers.rb actionpack-1.11.0/lib/action_controller/base.rb actionwebservice-0.9.3/lib/action_web_service/api.rb actionwebservice-0.9.3/lib/action_web_service/support/class_inheritable_options.rb actionwebservice-0.9.3/lib/action_web_service/base.rb actionwebservice-0.9.3/lib/action_web_service/dispatcher/abstract.rb actionwebservice-0.9.3/lib/action_web_service/protocol/soap_protocol.rb actionwebservice-0.9.3/lib/action_web_service.rb actionwebservice-0.9.3/Rakefile activerecord-1.13.0/lib/active_record/associations.rb activerecord-1.13.0/lib/active_record/fixtures.rb activerecord-1.13.0/test/class_inheritable_attributes_test.rb activesupport-1.2.3/lib/active_support/class_inheritable_attributes.rb activesupport-1.2.3/lib/active_support.rb facets-2005.10.30/lib/mega/inheritor.rb rails-0.14.3/lib/rails_generator/base.rb

Considering all the tech being built on top of these two projects, I'd hanker to guess that this alone consitutes nearly half the uses cases out there. And without a doubt these hacks are being done in other projects too.

Your #extend alternative misses the whole point. It thwarts encapsuation, as I have said, which is exactly why it is so rarely used. Why would anyone write a reusable module, or worse two modules as you suggest, and have to both include and extend, when the whole point is that they should always be together. Absoultely no one --hence bring in ClassMethods hacks.

You continue to speak theoretically. I'm showing you real practice. You've yet to show me reason for not having this grounded in actual code issues.

You, know this reminds me of something in college. Students would have to walk to the upperside of campus for classes, and there was this one area where they would cut right across a small field rather than take the L-shaped course along the sidewalk. No matter how much the administration tried to tell students they "should" stick to the sidewalk, they kept taking the path. After awhile the path became so well worn the administration gave up trying to defend what "should" be and put in a new sidewalk.

T.


I didn't say no one ever used inherit. (But weren't we talking about included?) I said I don't think that including a module should change the interface of the class or module object that's doing the including. If you need that functionality, it's available. There's no reason to pay a price to have it available redundantly.

Obviously David Hansson has some reason for structuring the code the way he does. It would be child's play to define all those methods directly on ActiveRecord::Base -- so it may be safely assumed that the fact that it isn't done that way is a design decision, not a hack.

Look at what's actually happening. He's using "ClassMethods" as a way of tagging methods and deferring their inclusion. The methods defined in the inner "ClassMethods" modules are *not* actually class/module methods of the module in which they're defined (so your 'included' thing doesn't play a role anyway); they're instance methods in a module, held in reserve to be added in bulk to an ancestral class. That's a design decision. For one thing, it quarantines these methods so that there's no danger that extra methods will spill over during an "extend" operation. It also has a kind of self-documenting quality.

I don't see how this example shows anything at all that would or should change if include started interfering with class methods. And if there are examples in "real practice" of the misuse of class/module idioms, then like all buggy code they should be fixed.

David


David, from your initial statement it's apparent you don't understand what's being done here. Moreover, and I'm sorry to have to say this, but trying to save face with DHH by stating his usecase a "design decision" and then to declare everyone else's identical practice a "misuse" and "buggy code" is unconscionable. I've looked at the DHH's code and there's nothing special about it. If you feel the practice is flawed, fine, say so. But don't play sides. If you wish not to consider practice at all and want grounding on theoretical arguments only, fine, say so as well, and we can discuss theory.


I can't follow you into the realm of "sides" and all that (it's not of interest to me), but on the technical side my point about buggy code was specifically in relation to your example -- the one with "CRASH" in it. Your point was that if you assume this technique works, your program will crash. My point is: showing a scenario where the mistaken assumption is made that a given technique will work doesn't mean that the language should be changed to make that technique work. It means that anyone whose code has that particular bug in it should fix it.

(It also doesn't mean that the language should *not* be changed. It just didn't seem relevant to that question.)

David


+1

This will also make documentation of modules much easier.

  module M
    # Returns the amount of "foo"
    def self.foo; end
  end

Cheers, Daniel Schierbeck


Daniel --

What exactly is that comment easier than? :-)

My problem with this whole thing is that I'm not seeing what the *exact* relationship is supposed to be between C and M in this example:

  module M
    def self.x
    end
  end
  class C
    include M
  end

Supposing that C now responds to "x" -- why? What's actually happened?

In the inheritance case:

  class C
    def self.x
  end
  class D < C
  end

what's happened is that the singleton class of C has an instance method "x", and since it is the superclass of the singleton class of D, the object D responds to "x".

In the proposed module behavior, what would the actual mechanism be? Right now it seems like: automatically erase the character sequence "self." and then magically extend the class's singleton class with what's left. I don't see what in the class/module model would make it make sense for a singleton method of a module to make this quantum leap out of itself and into an unrelated context.

I don't like the idea of something like this just being decreed to be how things work. I could imagine the model changing -- indeed, Matz has said that the whole singleton-class model is just one possible way to implement per-object behavior -- and maybe then there would be space for something like this. But as long as there *is* a particular mechanism, I don't like the idea of seeing it opted out of on an ad hoc basis.

David Black


@ david

It's easier than this:

  module M
    def self.included(klass)
      klass.module_eval do
        def self.foo
          "I'm a class method!"
        end
      end
    end
  end
  class Klass
    include M
  end
  Klass.foo -> "I'm a class method"

Compare that to this:

  module M
    def self.foo
      "I'm a class method"
    end
  end
  # ...

I just think it's more intuitive that when you include a module, you include *all* the behaviour defined in that module, *not* just the instance methods.

Cheers, Daniel Schierbeck


"Intuitive" is tricky at the best of times (learning things isn't *so* horrible :-) and here I don't think it comes into play much. There has to be something beyond a feeling that the word "include" should trigger a certain event. That's what I meant about some kind of reconciliation of this with the underlying model.

It's not a matter of "just" including the instance methods and somehow stopping short, for no reason, of finishing the job. Adding methods to a class's singleton class isn't part of the job; it doesn't fit. That's my problem with it. What are the *actual* implications and mechanisms, in terms of the method lookup path and how it's affected? And what's the logic connecting a module's singleton methods with a class's?

David


I rarely combine module methods and instance methods into the same module.

I never have had a desire for module methods to be callable on the class the module was included into.

It seems that the only reason Rails uses ::Methods is to have extended documentation on a particular set of class methods, not for reasons related to "missing features" in Ruby.


Dave, I'm not sure why you think it's so "magical". It would fit into the class model just as a module does on the instance plane, but on the class plane. Ie. mirror the proxy class with a proxy singleton that ties to the module's singleton.

T.


Oops, s/Dave/David/


> It seems that the only reason Rails uses :: is to have > extended documentation on a particular set of class methods, not > for reasons related to "missing features" in Ruby.

That's what David Black said and it simply not true. Why go o the trouble of using append_features just for that? Are you looking at more than activesupport?

T.


I think it's magical because I don't see how else to explain it. I don't even know what "it" is.

Once again:

  module M
    def self.x
    end
  end
  class C
    include M
  end

Remember: including a module does *not* add methods to *any* class (even though people sometimes describe it that way). It adds a *module* to a lookup path. That's a really important difference.

It seems to me that you're proposing that methods be added to C's singleton class when M is included. That just isn't what module inclusion does (nor what inheritance does; that's not the basis of C.x being visible to D when D < C).

What you have to figure out is: which module is being added to which lookup path? That's not clear, to me, from what you've said so far.

Since the method "x" is being defined in M's singleton class, there actually is *no* module (as opposed to class) that actually contains "x". "x" is defined in a class, and in a class only. The reason M can call it is that M's singleton class lies in M's method lookup path.

So you're basically saying: when a module M is mixed into a class C, M's singleton class should be automatically converted to a module (or duped into a module, I guess), and then placed in the lookup path of C object (i.e., included by its singleton class). Another interpretation would be that you're suggest that the methods in M's singleton class should be duped and stashed in C's singleton class.

None of this corresponds to the existing class/module/mixin/inheritance/... model(s). So what you're suggesting is a completely anomalous behavior. I understand that people who see the character sequence "self." and don't entirely grasp the model might think that they must be, or should be, creating class methods when they mix in a module. Pedantic as it may be, I'm in favor, as I've said, of having people learn what's actually going on, and not just chucking in the whole design for this kind of reason.

David


LOL "Chucking the whole design"? You're blowing this out of proportion. It doesn't chuck out the design at all, but fits right into it. (And I am well aware of how it works). The fact the modules singleton class is a class and not a module isn't of consequence. The distinction between Module and Class to begin with (and as you know) is largely arbitrary. I see no reason a class can't be tied in through proxy just as easily as a module.

If you want, I can draw you an ascii-art diagram of the class structure, but it woud be much eaiser if you could have a look at figure's 19.2 "Adding a metaclass to Guitar" and 19.4 "An included module and it's proxy class" in the Pickaxe. You can see where the proxy in 19.4 would fit into 19.2. To implement this proposal simply add a proxy class (a meta proxy) between the metaclass Gutair' and Object', which links back to the metaclass of the module. That's it. Like I said, fits right in, and IMO naturally completes it.

T.


I think we've reached the agree-to-disagree stage.

David


> I think we've reached the agree-to-disagree stage.

Okay then. But honestly I don't see why. No one has yet given a single concrete argument that counter the advantages of this proposal. All that has been offered thus far is essentially opinionation. Oh well.

T.


I vote no.

The reasons why:

1. I don't see a need for it. The current method may be a "hack", but it's fairly straight-forward, and it works. I don't see the _need_ for a "fix".

2. I like the forced seperation right now. As said, it has a nice self-documenting flair I like, and using the ClassMethods "hack" results in cleaner code.

What's not to like about it? That as a Ruby newcomer it wasn't entirely clear what the differences between how you could call/use module methods were? But I learned, and I can't honestly say that staying with the class paradigm is "better". Probably "simpler", but if simple was always better we wouldn't have the distinction between constructs such as Singleton or Static.

Is it a simple distinction? Or "intuitive"? Does it even serve any practical purpose 99.9% of the time? But it still has value, and I'd rather not see Ruby going for that level "simplicity" personally.


I think what amazes me about people voting NO is that not only have they've never needed the feature so thay have no idea why it's useful, but even more amazingly, because of this fact, even if the feature was implemeneted, they wouldn't even notice it was there! The feature has no significant counter effects, so someone who doesn;t ever use it wouldn't know it's there. The fact is even appearent in the arguments presented against it --they are all substanceless, speaking in nothing but theoreticals and "how I feels" --despite the plain practical value.


If your argument is valid, I am sure there will be more and more examples of the hacks you mentioned. Lets see what time brings and revive the suggestion at a later time? Impatience doesnt help ;)


It seems very clear cut to me: making the change would help people who are using class methods in modules and would almost certainly cause no harm to people who are not using them. From the discussion, it almost seems like the strongly opposed votes are out of spite. -elijah


We should note that a compromise solution has been tentively reached on this issue. For class methods to be "carried on" you will put them in a class_extension block. Eg.

  module M
    class_extension do
      def x; "x"; end
    end
  end
  class X
    include M
  end
  X.x #=> "x"


Strongly opposed 5
Opposed 2
Neutral 0
In favor 0
Strongly advocate 6
ruby picture
If you have registered at RCRchive, you may now sign in below. If you have not registered, you may sign up for a username and password. Registering enables you to submit new RCRs, and vote and leave comments on existing RCRs.
Your username:
Your password:

ruby picture

Powered by .