I agree with the first comment that this question is far too broad to be answered. But I'll try anyway.
There are several aspects to your question:
- What are the semantics of the commonly used concepts of object-oriented programming?
- How can they be implemented in a compiler?
- How are they usually implemented in other compilers?
- What are good resources for further studies?
Semantics
Varies widely between languages and there also is quite a bit of confusion/controversity about what OOP actually means (a nice presentation on that topic: http://www.infoq.com/presentations/It-Is-Possible-to-Do-OOP-in-Java which also has some examples of implementing OOP-features). Just pick one model and look up a reference that defines the semantics such as a language specification or a scientific paper on the model.
Javascript probably is the easiest model to implement as it very directly maps to the implementation without much of a necessary surrounding framework in the compiler. A static version of the Java model (compile time class compilation instead of runtime classloading) shouldn't be too hard either. More sophisticated models would be C++ (which allows multiple inheritance) and Smalltalk or Common Lisp/CLOS (with Meta-object-protocols).
Possible Implementations
Again a wide range of choices. As the semantics are fixed and mostly rather straightforward the implementation effort most strongly depends on the performance you want to archive and the existing infrastructure of your compiler. Storing everything in lists and scanning them for the first entry that satisfies the rules probably is the easiest implementation.
Usual Implementation
Most programming languages out of the Java/C#/C++ area do static compile-time name/signature lookups to find the definitions of the things referred to and use a http://en.wikipedia.org/wiki/Virtual_method_table to resolve polymorphic calls. They also use the Vtable pointer for instanceof
-checks and for checking down-casts.
Resources
While only 30 pages are directly concerned with objects I still think Lisp in Small Pieces (LiSP) is a great book for learning to work at that level within a compiler. It focusses on implementing language features, trade-offs in implementations and fitting the pieces together. (if (you can get over the the syntax used) (it's great)).