Assuming that the Hazelcast Enterprise API is an extension of the Hazelcast API, it might be as simple as recompiling the Openfire Hazelcast plugin with a different Hazelcast dependency.
I did a quick test. The plugin compiles just fine after you swap the dependency on Hazelcast with a dependency on the 'enterprise' variant, like this (your version number might vary):
<dependencies>
<dependency>
<groupId>com.hazelcast</groupId>
<artifactId>hazelcast-enterprise</artifactId>
<version>3.10.6</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>hazelcast</id>
<url>https://dl.bintray.com/hazelcast/release/</url>
</repository>
</repositories>
Most of the Hazelcast configuration can be done in the XML file that's already accessible as a stand-alone file in Openfire. There's a good chance that you don't need any code modifications to get things configured in the way you want.