I need to get data from a PostgreSQL DB and index it into Elasticsearch. https://www.elastic.co/blog/logstash-jdbc-input-plugin
When I run /opt/logstash-2.3.3/bin/logstash -v -f es_table.logstash.conf
I receive the following error:
Pipeline aborted due to error
{:exception=>#<LogStash::ConfigurationError: org.postgres.Driver not loaded.
Are you sure you've included the correct jdbc driver in :jdbc_driver_library?>, :backtrace=>["/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-3.0.2/lib/logstash/plugin_mixins/jdbc.rb:156:in `prepare_jdbc_connection'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-3.0.2/lib/logstash/plugin_mixins/jdbc.rb:148:in `prepare_jdbc_connection'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-input-jdbc-3.0.2/lib/logstash/inputs/jdbc.rb:167:in `register'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:330:in `start_inputs'", "org/jruby/RubyArray.java:1613:in `each'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:329:in `start_inputs'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:180:in `start_workers'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/pipeline.rb:136:in `run'", "/opt/logstash-2.3.3/vendor/bundle/jruby/1.9/gems/logstash-core-2.3.3-java/lib/logstash/agent.rb:473:in `start_pipeline'"], :level=>:error}
Here is a piece of my Logstash configuration:
input {
jdbc {
jdbc_user => 'user'
jdbc_driver_class => 'org.postgresql.Driver'
jdbc_connection_string => 'jdbc:postgresql://1.1.1.1:5432/db'
lowercase_column_names => false
clean_run => false
jdbc_driver_library => '/usr/share/java/postgresql-jdbc4.jar'
jdbc_password => 'pass'
jdbc_validate_connection => true
jdbc_page_size => 1000
jdbc_paging_enabled => true
statement => 'SELECT * FROM "table"'
type => 'table'
}
...
The jdbc4 driver exists. I tried jdbc3 too without success.
ls /usr/share/java | grep postgresql-jdbc
postgresql-jdbc3-9.2.jar
postgresql-jdbc3.jar
postgresql-jdbc4-9.2.jar
postgresql-jdbc4.jar
The Driver class is inside:
jar tf /usr/share/java/postgresql-jdbc4.jar | grep -i driver
org/postgresql/Driver$1.class
org/postgresql/Driver$ConnectThread.class
org/postgresql/Driver.class
org/postgresql/util/PSQLDriverVersion.class
META-INF/services/java.sql.Driver
The port 5432 is open:
telnet 192.168.109.108 5432
Trying 192.168.109.108...
Connected to 192.168.109.108.
Escape character is '^]'.
Authentication to the DB works.