Storage Capacity Issue/Errors Datastax Custom auto-cluster EC2 AMI

Question

I am trying to create a custom auto-clustering AMI to be used with the Datastax Enterprise 3.0 OpsCenter create cluster wizard.

I've tried at least two methods and keep ending up at the same place. When I use the standard Datastax auto clustering (AC) AMI the size of the cluster is in the terabytes. When I used my custom ami the size of the cluster is 40GB. I am selecting 4 m1.extra large instances for each launch.

Method 1: I took the standard datastax AC AMI made my modifications used the instruction set here: https://github.com/riptano/ComboAMI/blob/2.4/presetup/setup.md (stopping at "# Setup credentials") I then used the following to bundle and upload to S3 ec2-bundle-vol -k -c -u -e ec2-upload-bundle -b -m -a -s After that I used the EC2 Management console to register the AMI via the image.manifest.xml file.

Method 2: I used the Ubuntu 12.04 LTS AMI instance made my modifications and used the same instruction set above.

Both methods gave me a 40GB cluster. It looks like the custom AMIs do not set up the raid correctly or at all.

df -h for custom AMI

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      9.9G  2.6G  6.9G  28% /
udev            7.4G  8.0K  7.4G   1% /dev
tmpfs           3.0G  200K  3.0G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/xvdb       414G  199M  393G   1% /mnt

df -h for default AMI

Filesystem      Size  Used Avail Use% Mounted on
/dev/xvda1      9.9G  2.2G  7.3G  23% /
udev            7.4G  8.0K  7.4G   1% /dev
tmpfs           3.0G  224K  3.0G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.4G     0  7.4G   0% /run/shm
/dev/md0        1.7T   34M  1.7T   1% /raid0

joaquin joaquin · Accepted Answer · 2013-11-25T23:13:56

I understand this is quite old, but if you follow the exact commands it should allow you bundle the AMI without issues since this is the exact same script we use to bundle our AMI.

If the commands are not the culprit, you should look at the ~/datastax_ami/ami.log. If there are any ERROR statements, those will probably tell you what's wrong. A typical case is where a small syntax issues causes only part of the script to be run.

And from the looks of it, since you don't have a /raid0 mount point and instead have a /mnt mount point, something probably went wrong with your script before it had a chance to run completely.

Storage Capacity Issue/Errors Datastax Custom auto-cluster EC2 AMI

1 Answers