The answers can be found in the latest EJB Specification.
1) Does container implement the JTA spec for transaction handling?
21.3.3 JTA 1.1 Requirements
The EJB container must include the JTA 1.1 extension, and it must
provide the javax.transaction.UserTransaction interface to enterprise
beans with bean-managed transaction demarcation through the
javax.ejb.EJBContext interface, and also in JNDI under the name
java:comp/UserTransaction, in the cases required by the EJB
specification. The other JTA interfaces are low-level transaction
manager and resource manager integration interfaces, and are not
intended for direct use by enterprise beans.
2) Does Container provided JTS service?
13.1.3 Relationship to JTA and JTS.
The EJB architecture does not require the EJB container to support the
JTS interfaces. The EJB architecture requires that the EJB container
support the JTA API defined in [8] and the Connector APIsdefined in
[15].
3) If #1 and #2 aren't true then how does container provide the transaction support? Does it "borrow" functionality from some other open source project for doing these kind of work?
I really don't get this questions. In general Oracle/Sun provide a set of interfaces or specification, each vendor (Glassfish, Weblogic, JBoss, etc...) will then provide its concrete implementation.
In the past, an application server had to be certified by Sun to be considered JEE compliant, I suppose it is exactly the same with Oracle. So, you can provide concrete implementation by means of open source libraries, someone else will decide if your server is compliant.