Why does the EC2 instance not start correctly after resizing the root volume of the EBS?

Question

I used the instructions https://matt.berther.io/2015/02/03/how-to-resize-aws-ec2-ebs-volumes/ and http://atodorov.org/blog/2014/02/07/ aws-tip-shrinking-ebs-root-volume-size / to go to an EBS volume with less disk space. In both cases, when I attached the compressed volume of EBS (like / dev / xdva or / dev / sda 1, none of them work) to the EC2 instance and run it, it stops with the message itself

State transition reason Client.InstanceInitiatedShutdown: Instance initiated shutdown 

A few more wizards, and I found that the new volume does not have a BIOS boot partition. So I used gdisk to create one and copied the MBR from the original volume (which works and with which I can run instances) to the new volume. Now the instance does not complete, but I cannot ssh into the recently launched instance.

What could be the reason for this? How can I get more information (from logs / AWS Console, etc.) about why this is happening?

Answer 1

To reduce the size of the bootable ETS disk with GPT loading below 8 GB that seem to use standard images, you can do the following: (slight deviation of the dd method from https://matt.berther.io/2015/02/03/ how-to-resize-aws-ec2-ebs-volumes / )

source disk /dev/xvdf , target /dev/xvdg

• Compress the original partition

   $ sudo e2fsck -f /dev/xvdf1 $ sudo resize2fs -M /dev/xvdf1 

  Will output something like

   resize2fs 1.42.12 (29-Aug-2014) Resizing the filesystem on /dev/xvdf1 to 257491 (4k) blocks. The filesystem on /dev/xvdf1 is now 257491 (4k) blocks long. 

  I converted this to MB, i.e. 257491 * 4/1024 ~ = 1006 MB

• copy above size + a little more from device to device (!), and not just partition, as this includes both the partition table and the data in the boot partition

   $ sudo dd if=/dev/xvdf of=/dev/xvdg bs=1M count=1100 

• now use gdisk to fix the GPT partition on the new drive

   $ sudo gdisk /dev/xvdg 

  You will be met approximately

   GPT fdisk (gdisk) version 0.8.10 Warning! Disk size is smaller than the main header indicates! Loading secondary header from the last sector of the disk! You should use 'v' to verify disk integrity, and perhaps options on the experts' menu to repair the disk. Caution: invalid backup GPT header, but valid main header; regenerating backup header from main header.# Warning! One or more CRCs don't match. You should repair the disk! Partition table scan: MBR: protective BSD: not present APM: not present GPT: damaged **************************************************************************** Caution: Found protective or hybrid MBR and corrupt GPT. Using GPT, but disk verification and recovery are STRONGLY recommended. **************************************************************************** Command (? for help): 

  Below is the keyboard input in gdisk . To fix the problems, the data partition that is present in the copied partition table must be modified to fit the new disk. This means that it needs to be recreated less, and its properties must be set in accordance with the definition of the old section. I have not tested it, so it may not be necessary to move the backup table to the actual end of the disk, but I did it anyway:

  • go to additional expert options: x
  • Move backup data structures to the end of the disk: e
  • return to the main menu: m

  Now, to fix the partition size

  • print and mark some properties of section 1 (and other non-reloadable sections, if they exist):
    i
1
    Show something like

     Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem) Partition unique GUID: DBA66894-D218-4D7E-A33E-A9EC9BF045DB First sector: 4096 (at 2.0 MiB) Last sector: 16777182 (at 8.0 GiB) Partition size: 16773087 sectors (8.0 GiB) Attribute flags: 0000000000000000 Partition name: 'Linux' 

  • delete d now
    1
    and recreate the section
    n
1
    Enter the required parameters. All default values ​​worked for me here (= press enter), if in doubt, refer to the information in the section above

    • First sector = 4096
    • Last sector = whatever the actual end of the new disk is - use the default here
    • type = 8300 (Linux)

  • The new default name for the section does not match the old. So change it to the original One c
1
Linux (see Partition name above)

  • The next thing to change is the section GUID
    x
c
1
DBA66894-D218-4D7E-A33E-A9EC9BF045DB (see Partition unique GUID , not the section code above)

  • It should be like that. Back to main menu and print status
    m
i
1
    Now print

     Partition GUID code: 0FC63DAF-8483-4772-8E79-3D69D8477DE4 (Linux filesystem) Partition unique GUID: DBA66894-D218-4D7E-A33E-A9EC9BF045DB First sector: 4096 (at 2.0 MiB) Last sector: 8388574 (at 4.0 GiB) Partition size: 8384479 sectors (4.0 GiB) Attribute flags: 0000000000000000 Partition name: 'Linux' 

    The only change should be Partition size .

  • burn to disk and exit w
y

• grow a file system to fit the entire (smaller) disk. The fist step reduced it to the minimum size that it can fit

   $ sudo resize2fs -p /dev/xvdg1 

• We are done. Disconnect the volume and remove it.

• Extra step. Choosing the right kernel ID for AMI.

If you are dealing with a PVM image and encounter mount error in instance logs

Kernel panic - not synchronization: VFS: root cannot be installed

when your instance fails the startup check, you may need to complete this additional step.

The solution to this error was to select the correct kernel ID for your PVM image while creating the image from the snapshot. A complete list of kernel identifiers (AKIs) can be found here .

Make the right AKI for your image, they are limited by areas and architecture!

Answer 2

The problem was in the BIOS boot partition. I was able to solve this by first initializing an instance with less EBS. Then disconnect the volume and bind it to the instance that will be used to copy content from a higher volume of a lower volume. This created a bootable BIOS partition that really works. Simply creating a new one and copying the boot partition does not work.

Now, following the steps described in either of the two links, you can reduce the amount of root EBS.