Correct way to handle automatic mounting of volumes when restarting instances
Background#
- Operating System: Amazon Linux 2023
- Instance Type: t2.micro
I wanted to mount a new volume to my EC2 instance, but when I restarted the instance, the mounted volume disappeared. So, I searched online for how to automatically mount it upon reboot. I tried modifying the /etc/fstab file and then restarted, but I couldn't connect via SSH. After checking the logs, I realized that I had entered emergency mode... I couldn't even access the interactive interface. "Clearly, I messed up the first time."
Initially, I thought it was a problem with my proxy, but even after disabling it, I still couldn't connect.
I checked the system logs.
But AWS also provided a solution for failed startups, and there was a small line of text below.
Result:
Initially, I kept looking at the documentation, trying to find out which type of instance would support it, and even tried changing the instance type, but for some reason, I still couldn't connect... Finally, I found a method to recover by temporarily starting a new instance for recovery and modifying the damaged file.
Recovery Method#
Basically, it involves the following steps:
- Stop the failed instance.
- Detach the root volume from the failed instance.
- Create a new EC2 instance in the same availability zone.
- Attach the root volume to the new instance.
- Modify the
/etc/fstabfile.
Stop the Instance#
Just wait for it to stop. The purpose is to detach the volume.
Detach the Volume#
Find the volume mounted on the root ("/") directory and click on "Detach".
Start a New Instance#
Configure it with the simplest settings, as the purpose is just to attach the previously detached volume.
Remember to specify the subnet (to select the same availability zone as the volume, for example, mine is us-west-1b).
Attach the Volume#
In the AWS management console, attach the previously detached volume to the new instance, and SSH into the new instance to mount it.
Here, you can see the content that we mistakenly modified earlier.
Use vim to directly modify the fstab file. After making the necessary changes, detach the volume and attach it back to the previously non-bootable instance. Remember to enter "xvda" as the name when attaching it back, as that was the default name I saw for the root volume in AWS.
Finally, start the old instance, and the newly created instance can be deleted.
Success!