Database data integrity issue

Question

I need to make a windows service(process data every 5 mins) that

takes each record from Table1(Queue table) (MSSQL)
insert that record into various tables Table2, 3, 4, 5 (MySQL) (with Engine MyISAM)
insert data from Table1 and Table2 into Table3(Mapping table) (MSSQL)
delete the record from Table1 again.

How can this be done effectively as it seems MySQL doesn't support transactions. Can I use TransactionScope? Or shall I manually delete the records created in previous step if error occurs in next step. I am using MySQL Connector and Linq2SQL.Any suggestion will be helpful. Thanks.

Johan Johan · Accepted Answer · 2011-06-23T09:27:21

MySQL does support transactions, but not for MyISAM, you'll have to use InnoDB or likewise engine if you want transactions.

There is a trick you can use and still keep MyISAM and have transactions (kind of).
Here's how to have your cake and eat it too :-).

Step 1
Create a blackhole table

CREATE TABLE bh_insert_tables 
   t1.pk integer,
   t2field1 varchar(45),
   t2field2 integer,
   ....
   t3field1 integer,
   .... etc for all tables
   ) ENGINE = BLACKHOLE;

Step 2
Create a memory table to store the dummy transactions into

CREATE TABLE my_rollback
   id unsigned integer auto_increment primary key,
   last_insert integer not null,
   tablename varchar(15) not null,
   index last_insert using hash ('last_insert'),
   index tablename using hash ('tablename') 
) ENGINE = MEMORY;

CREATE TABLE status
  id unsigned integer auto_increment primary key,
  insert_id integer not null,
  success boolean not null,
  index insert_id using hash ('insert_id')
) ENGINE = MEMORY;

Step 3
Put a trigger on the blackhole table that will fire on insert.
This trigger will also add transaction support (kind of) to MyISAM.

DELIMITER $$

CREATE TRIGGER ai_bh_insert_each AFTER INSERT ON bh_insert FOR EACH ROW
BEGIN
  DECLARE table1ID integer;
  DECLARE error boolean;

  DECLARE CONTINUE HANDLER FOR SQLEXCEPTION
  BEGIN
     SET error = TRUE;
  END;

  SET error = FALSE;

  INSERT INTO table2 (field1, field2, field3) VALUES (NEW.t2field1, NEW.t2field2, NEW.t2field3);
  IF not(error) THEN BEGIN  
    INSERT INTO my_rollback (last_insert, tablename) VALUES (LAST_INSERT_ID(), 'table2');
    INSERT INTO table3 (field1,field2) VALUES (NEW.t3field1, NEW.t3field2);
  END; END IF;
  IF NOT(error) THEN BEGIN
    INSERT INTO my_rollback (last_insert, tablename) VALUES (LAST_INSERT_ID(), 'table3');
    INSERT INTO table4 .......
  END; END IF;
  IF error THEN BEGIN  /*do_rollback*/ 
    DELETE table2 FROM table2
    INNER JOIN my_rollback ON table2.id = my_rollback.last_insert
    WHERE my_rollback.tabelname = 'table2';

    DELETE table2 FROM table2 .......
    INSERT INTO status (insert_id, success) VALUES (NEW.pk, false);
  END; 
  ELSE BEGIN
    INSERT INTO status (insert_id, success) VALUES (NEW.pk, true);
  END; END IF;
  /*Clear my_rollback for the next insert*/
  DELETE FROM my_rollback WHERE id IS NOT NULL;
END $$

DELIMITER ;

In your insertion routine you can query table status to see which records from table1 in MSSQL where inserted successfully and which ones failed.

Links:
http://dev.mysql.com/doc/refman/5.0/en/blackhole-storage-engine.html
http://dev.mysql.com/doc/refman/5.0/en/triggers.html
http://dev.mysql.com/doc/refman/5.5/en/create-trigger.html
http://dev.mysql.com/doc/refman/5.5/en/delete.html
http://dev.mysql.com/doc/refman/5.5/en/declare-handler.html

Database data integrity issue

2 Answers