The ActiveMQ component, as well as the JMS component, is based on JMSTemplate from Spring Framework, when sending messages.
As you fear, it creates and closes a connection+session+producer per message, which is not optimal performence wise.
As recommended in the activemq component documentation, you should set it up with pooling. Then you will reuse the connection/session/producer.
You could actually live with just one connection in the pool for that pool if you intend to use it for this route only.
ActiveMQ has made a summary of tips and tricks with JMSTemplate. It's worth reading.
Update: Regarding the number of Connections related to concurrency:
Simplified, a Connection is a network connection ("socket") with the broker, that is shared among many threads. Each of your threads would still have it's own (pooled) session that uses the shared connection and operate concurrently. Multiple connection could still be an enhancement, if you have multiple ActiveMQ brokers and want to loadbalance between them (given ActiveMQ is the bottleneck and not your application).
failover:(ssl://host1:61616,ssl://host2:61616)
It might also be that, in practice, performance might go up to some small extent by using a limited set of connections instead of one, but that you probably have to test out for your particular environment.