I am playing with mysql and vertica and have a master/slave mysql pair that is running with mixed binlog format (default settings).
To replicate this to the vertica database for some analysis tasks, I was considering tungsten.
The master is busy with read/write requests from the app, and the slave is just recording whats happening, so ideally I want to run the tungsten replicator on the mysql slave as it is only reading as well.
Tungsten requires the binlog to be in row based format, and so I enabled a row based binlog on the slave, which creates the binlog and the relay-binlog files. So the slave reads mixed binlog from the master, applies to its database and also keeps the changes in the relay-binlog in row based format. Everything fine so far.
When I use the standard -a --master-slave argument it appears to be only looking at the binlogs that only contain things that happened locally on the slave, but not the relay-binlogs.
The documentation and posts I found all point to running tungsten on the master, and to use it for replication throughout without using mysql master/slave, rather than that hybrid mode I have in mind.
However I would prefer to keep whats working and just add tungsten as a bridge replicating from mysql slave to vertica.
How would I configure that?
How do I tell it what tables I want to replicate from the slave to vertica, rather than replicate all?
Configuration detail:
/etc/my.cnf of slave:
datadir=/var/lib/mysql
log-bin=/var/log/mysql/mysql-bin.log
expire_logs_days=7
# for tungsten replicator we need row based binlogs on the slave
binlog-format=row
## Server timezone is GMT.
#default-time-zone='+00:00'
# Tables default to UTF8.
character-set-server=utf8
collation-server=utf8_general_ci
server-id=3
The script I use to stage tungsten:
tungsten-replicator-2.1.2-30/tools/tungsten-installer --master-slave -a \
--service-name=mysql2vertica \
--master-host=testslave1 \
--cluster-hosts=testslave1 \
--datasource-user=tungsten \
--datasource-password=supersecret \
--home-directory=/opt/tungsten \
--buffer-size=1000 \
--java-file-encoding=UTF8 \
--java-user-timezone=CDT \
--mysql-use-bytes-for-string=false \
--svc-extractor-filters=colnames,pkey \
--property=replicator.filter.pkey.addPkeyToInserts=true \
--property=replicator.filter.pkey.addColumnsToDeletes=true \
--start-and-report
It seems to start fine, says its online, but only copies what is in binlog but not relay-binlog.