6
votes

We got MySQL database on AWS RDS with innodb engine, the MySQL version is 5.6.19.

When trying to add a column in a table, we get the error message below:

ERROR 1041 (HY000): Out of memory; check if mysqld or some other process uses all available memory; if not, you may have to use 'ulimit' to allow mysqld to use more memory or you can add more swap space

The script we run to alter table is below: ALTER TABLE mytablename ADD COLUMN temp_colume varchar(255) NULL AFTER temp_firstcolumn;

Our RDS is on db.m3.2xlarge with 30GB memory: Our innodb buffer size is DBInstanceClassMemory*3/4 ~= 24GB

We can successfully re-creating the table with the column changes already made to it, but we are getting error when altering tables.

Does anyone meet the same issue?

2
Is there possibly a lot of updates on the table while you're doing the ALTER TABLE?Julie Pelletier
I also checked the SHOW PROCESS LIST which shows the DB is not busyI.83
How about ulimit -a from the process that starts the MySQL daemon? Did you look at the memory usage as it was processing, before it failed?Julie Pelletier
If you did not change any settings in my.cnf, you should write a bug with RDS, claiming that their default settings can lead to "out of memory".Rick James

2 Answers

7
votes

I've seen inplace alters in RDS fail lately. AWS support recommended modifying the alter table statement to look like this:

ALTER TABLE tbl ADD COLUMN abc varchar(123) AFTER zyx, ALGORITHM=COPY

The secret is to add

, ALGORITHM=COPY

to the end as a work around.

You could also failover the RDS instance https://dev.mysql.com/doc/refman/5.7/en/alter-table.html

2
votes

The error was fixed by simply rebooting our RDS. After rebooting, the memory increases 1.5 GB, we were have freeable memory ~ 3.5 GB, and now it is nearly ~5 GB. I am guessing the RDS itself (OS) cached some of the memory, but still a bit confused that why it gave error message when having 3.5 GB free memory and the table we tried to alter that was just 16KB.

Also, I find another similar issue. The link is below: https://dba.stackexchange.com/questions/74432/mysql-rds-instance-eating-up-memory-and-swapping