Configuring Failover cluster between 2 EC2 instances is bit tricky.
We can achieve Failover between 2 EC2 by configuring Elastic IP Failover.
In AWS we can associate Elastic IP to an EC2 instance. Using this Elastic IP we can connect to the associated instance using terminal. We can configure basic Failover of EIP using below steps. It is not recommended for Production purpose but still can be used.
Basic architecture of Failover of Elastic IP between 2 EC2 instances:
Steps to set failover between two EC2 instances:
Step1: Create IAM role name “HA”. It will allow our script to add or remove Network Interface from associated ec2 instances.
Attach below Policy to it.
{ "Version": "2014-12-17", "Statement": [{ "Effect": "Allow", "Action": [ "ec2:DescribeInstances", "ec2:AssignPrivateIpAddresses" ], "Resource": [ "*" ] } ]}
Step2: Launch 2 Instances and attach newly created IAM role
Step3: Install aws tools or aws-cli. In my case I have installed it in /opt/aws inside both servers.
Step4: Assign Secondary Private IP address to any of the instances eg. Server1. Then login to Server1 and use below command to activate secondary Private IP.
ifconfig eth0:0 <SecondaryPrivateIP> netmask <NetMask> up
In our example:
ifconfig eth0:0 172.16.0.3 netmask 255.255.255.240 up
Step5: Allocate Elastic IP and associate it with secondary private IP address of Server1 i.e. 172.16.0.3
Step6: Create scripts which will monitor EIP and shift EIP if any server goes down.
Here are basic scripts on Server1:
1. monitor.sh
#!/bin/bash
Node_IP=172.16.0.2
VIP=172.16.0.3
REGION=AWS region
. /etc/profile.d/aws-apitools-common.sh
Inst_ID=`/usr/bin/curl –silent http://169.254.169.254/latest/meta-data/instance-id`
EN_ID=`/opt/aws/bin/ec2-describe-instances $Inst_ID –region $REGION | grep eni -m 1 | awk '{print $2;}'`
echo "Starting HA monitor"
while [ . ]; do
pingresult=`ping -c 3 -W 1 $HA_Node_IP | grep time= | wc -l`
if [ "$pingresult" == "0" ]; then
echo `date` "– HA heartbeat failed, taking over VIP"
/opt/aws/bin/ec2-assign-private-ip-addresses -n $EN_ID –secondary-private-ip-address $VIP –allow-reassignment –region $REGION
ifconfig eth0:0 $VIP netmask 255.255.240.0 up
pingresult=`ping -c 1 -W 1 $VIP | grep time= | wc -l`
if [ "$pingresult" == "0" ]
echo "Restarting network"
/sbin/service network restart > /dev/null
fi
sleep 60
fi
sleep 2
done
2. /etc/profile.d/aws-apitools-common.sh
#!/bin/bash export AWS_PATH=/opt/aws export PATH=$PATH:$AWS_PATH/bin if [ -d /usr/lib/jvm/java ]; then export JAVA_HOME=/usr/lib/jvm/java elif [ -d /usr/lib/jvm/jre ]; then export JAVA_HOME=/usr/lib/jvm/jre fi
for aws_product in $(find /opt/aws/apitools /opt/aws/amitools -maxdepth 1 -type l); do [ -e $aws_product/environment.sh ] source $aws_product/environment.sh done unset aws_product export EC2_URL=https://ec2.us-east-1.amazonaws.com export AWS_CREDENTIAL_FILE=/root/.aws/credentials export EC2_HOME=/opt/aws/ec2
3. /root/.aws/credentials
[default] aws_access_key_id = ACCESS KEY aws_secret_access_key = SECRET ACCESS KEY;
Step 7. Run monitor.sh
./monitor.sh
Please follow above steps for Server2 as well.
On Server2 monitor.sh will keep monitoring Server1. And if Server1 shutsdown then it will shift ElasticIP to Server2. Same from Server1 side.
If you are using CentOS 6.5 or earlier version then you can put monitor.sh /etc/rc.local to run at boot time.
However if you are using CentOS 7 which uses systemd scripts then follow below process for enabling boot time execution of monitor.sh
– Place monitor.sh under /bin
– Create systemd-service for monitor.sh.
vi /usr/lib/systemd/system/monitor.service [Unit] Description=Monitor VIP After=syslog.target network.target auditd.service [Service] ExecStart=/bin/monitor.sh KillMode=process [Install] WantedBy=multi-user.target
– Enable monitor service and start it.
systemctl enable monitor systemctl start monitor
