I think JSNI code is slower in development mode. However, dev mode is apparently supposed to be replaced with super draft mode soonish, see https://plus.google.com/110412141990454266397/posts/iqXo5AyHkyd
Also, I am not sure to which extent GWT is able to analyze and optimize pure Javascript code. When compiling pure Java, GWT does a lot of inlining and other optimizations.
Another issue may be the extra round trip for loading the libraries.
Depending on the complexity of the library API (i.e. if you would need to wrap tons of classes), it may be simpler to convert a comparable pure Java library to GWT. But that obviously does not work for UI libraries or other libraries heavily depending on parts of the Java runtime not covered by GWT.
So if there are existing mature Javascript libraries that fit your purpose, it is probably best to wrap them in JSNI and go ahead as you suggest.
The JSNI boundary itself does not create any extra runtime overhead.