Home     |     .Net Programming    |     cSharp Home    |     Sql Server Home    |     Javascript / Client Side Development     |     Ajax Programming

Ruby on Rails Development     |     Perl Programming     |     C Programming Language     |     C++ Programming     |     IT Jobs

Python Programming Language     |     Laptop Suggestions?    |     TCL Scripting     |     Fortran Programming     |     Scheme Programming Language


 
 
Cervo Technologies
The Right Source to Outsource

MS Dynamics CRM 3.0

Python Programming Language

New-style classes and special methods


Hi

My question is about how special methods are stored internally in  
Python objects.
Consider a new-style class which implements special methods such as  
__call__ and __new__

class C(type):
        def __call__(...):
                <body>

class B:
        __metaclass__ = C
        <stuff>

b= B()

The type of C is 'type', that of B is 'C'. When B is instantiated,  
the __call__ method of C is first invoked, since C is the metaclass  
for B.

Internally, when a Python callable object 'obj' is called, the actual  
function called seems to be
'obj->ob_type->tp_call'.

Does this that somehow the '__call__' method defined in C above is  
assigned to the 'tp_call' slot in the object representing the class  
C, instead of it just being stored in the dictionary like a normal  
attribute? Where and how does this magic happen exactly? I'd  
appreciate any level of detail.

Thanks!
Raj

Yes, special methods populate the slots in the structures which Python
uses to represent types.  Objects/typeobject.c in the Python source
distribution does the hard work, particularly in function type_new (line
1722 in my current SVN checkout).

If you're not comfortable reading C code you may want to try looking at
the "Python implemented in Python" project, pypy, or perhaps
alternatives such as Jython (in Java) or better IronPython (in C#), but
I am not familiar in detail with how they deal with the issue.

Alex

 > Yes, special methods populate the slots in the structures which  
Python
 > uses to represent types.  Objects/typeobject.c in the Python source
 > distribution does the hard work, particularly in function type_new

Thanks for that quick response. I am quite comfortable with C code  
and am trying to understand exactly what happens when a new-style  
class is created, and then instantiated.

I have been reading typeobject.c and type_new() inside it in detail,  
and there are a few issues I am trying to figure out.

I can see a lot of *SLOT() macros in the file that seem to set the  
slots to appropriate values. What I am having trouble figuring out is  
the connection i.e. at what point during construction of the class  
object in type_new() are those slots allotted? Is it the tp_alloc()  
function which does this?

Is there some kind of descriptor or other mechanism connecting  
special method names with their slots in the object representation?  
(e.g. "__call__" with type->tp_call)

Also, what happens when a class inherits from multiple classes with  
their own __call__ methods? Where and how  is it decided which  
__call__ goes into the tp_call slot?

I'm sure I'll eventually figure it out if I stare at the code hard  
enough, but would totally appreciate any help I can get :)

Thanks again!

Raj

I believe there are different times -- one for the fundamental struct
that represents all types, others for secondary structs that represent
e.g. numerical/arithmetic methods, sequence methods, etc, each of which
is present only if necessary for a given type.

> Is there some kind of descriptor or other mechanism connecting  
> special method names with their slots in the object representation?  
> (e.g. "__call__" with type->tp_call)

The online docs for the C/API interface do some effort at explaining the
type-struct and auxiliary ones, and I spent a couple of pages on that
same subject in "Python in a Nutshell", though nowhere close to doing it
justice (such docs normally address people who want to write C-coded
extensions, rather than ones whose goal is to understand the existing
Python runtime -- though many of the issues are identical).

> Also, what happens when a class inherits from multiple classes with  
> their own __call__ methods? Where and how  is it decided which  
> __call__ goes into the tp_call slot?

The MRO is used (that stands for Method Resolution Order, and is also
implemented in the same C file, and documented there AND in a strong
essay by M. Simionato whose URL is, I believe, also in a comment in that
file) to find the relevant implementation.

> I'm sure I'll eventually figure it out if I stare at the code hard  
> enough, but would totally appreciate any help I can get :)

> Thanks again!

You're welcome, but I hope somebody else will also step up to offer
useful comments, because (what between my talk at Google Developer's Day
tomorrow, and traveling next week to speak in Krakow and then at the
first Italian conference on Python in Florence) I'm rather overwhelmed
these days;-).

Alex

The place to start is the PyType_Type tp_new slot function type_new().
The second to last statement is a call to fixup_slot_dispatchers(). This
function goes through the dictionary looking for special methods and
adds the appropriate slot functions. This gets very involved. I had to
use the Visual Studio debugger to follow what was happening when trying
to figure out what happens when assigning a special method to a class
after it is declared.

> Is there some kind of descriptor or other mechanism connecting special
> method names with their slots in the object representation? (e.g.
> "__call__" with type->tp_call)

There is a special tp_call slot function, slot_tp_call(), that calls the
user defined __call__. The same goes for other special methods.

Descriptors only come into play with extension types. In this case if a
slot function is found a descriptor is added to make the slot function
accessible from Python as a special method.

> Also, what happens when a class inherits from multiple classes with
> their own __call__ methods? Where and how  is it decided which __call__
> goes into the tp_call slot?

As Alex Martelli mentioned, __call__ is found using the method
resolution order. The tp_call slot function slot_tp_call() uses
lookup_method(), a variation of PyObject_GetAttribute(), to finding the
appropriate Python method. Its all documented in the C file.

> I'm sure I'll eventually figure it out if I stare at the code hard
> enough, but would totally appreciate any help I can get :)

Just ask.

--
Lenard Lindstrom
<l@telus.net>

Add to del.icio.us | Digg this | Stumble it | Powered by Megasolutions Inc