AWS hijinx: Windows instances launched from AMI fail to get new password, password recovery only makes things worse

More Windows, more password problems

When a Windows Server 2008R2 instance is launched from a custom AMI, the Amazon EC2Config service is supposed to generate a new random password for the administrator account. Instead when trying to use the “Get Windows Password” option. the user is presented with the dreaded “Your password is not Ready” error shown below:

What does this error mean? Well in a nutshell the EC2Config service did not generate a new password for this instance, and as a result the AWS console doesn’t have the new password to show you. This message always shows up immediately after a new windows instance is launched, but normally goes away after 15 minutes when the EC2Config service generates the new password, but if that new password is never generated you will still get this same error months after launching the instance.

Ok, that looks easy, lets fix it

A quick google search will tell you that the EC2Config service needs to be given permission to create the new password, and all will be right with the universe. This is done by editing the file “C:Program FilesAmazonEc2ConfigServiceSettingsconfig.xml”, to change the value of “Ec2SetPassword” to “Enabled”. Since you can’t login to the instance to allow this, you try the next search result which tells you to detach the root volume of the problem instance, and reattach to a windows instance which you can login to. Then you can mount the volume, edit the config.xml file, detach the volume, reattatach to the problem instance and then boot. Simple right?

Not so fast! You now find yourself with an instance that starts, but doesn’t boot anymore. From the AWS console you will only know that the instance fails the instance status check, while passing the system status check. If you could see the console of this instance, you’d see something like this:

Windows is telling you it can’t find a bootable partition, and is encouraging you to load the setup disc in the drive and perform a repair. I guess if I can find my AWS DVD drive, I can just… awww crud!

So what do we do here? My boot drive which was working before is now not booting, and even when it was I couldn’t login because I didn’t know the password. I’m sure the Cloud OPS team at Microsoft planned to fix this bug, but the fix it never made it into the last beta.

Taking the long way home

All is not lost, your volume still contains all of your data, and your instance still can boot from it, as long as it were marked as bootable. For some reason the alternate Windows instance to which this volume was attached unmarked the bootable flag, and then treated it as a secondary volume. This renders is useless as a boot volume without some additional magic.

What you need to do is reattach the volume to your alternate instance, and run a few handy utilities to reset the bootable flag on your volume. First utility is “bootsect.exe”, this is found on the windows 2008R2 setup disc in the boot folder. The other two “bcdboot.exe”and “bcdedit” are part of the standard windows installation, so you don’t need to go looking for them.

Assuming that you are:

  1. running these commands on an alternate Windows 2008R2 instance
  2. trying to fix a boot volume for Windows 2008R2,  mounted on the D drive
  3. rot running some exotic setup where windows was not installed on the boot drive of your problem instance

You need to run the following commands on your alternate instance to fix the boot volume for your problem instance:


bootsect /nt60 D: /mbr
C:windowssystem32bcdboot.exe D:Windows /s D:
bcdedit /store D:BootBCD /set {default} device partition=D:

bcdedit /store D:BootBCD /set {default} osdevice partition=D:
bcdedit /store D:BootBCD /set {bootmgr} device partition=D:

Each command should return a message indicating success. Once this is done, you can reattach the volume to your problem instance, and it should boot normally and reset the administrator password. You should now be able to get the new password from the AWS console and login to your instance.

Was all that really necessary?

No, it wasn’t. Why all this information explaining how to do something you don’t need to do? Well, the problem is that by the time you realized you couldn’t login to the new instance, you had already made a mistake. The way to avoid this problem entirely is to tell the EC2Config service it can reset your password before you create the AMI in the first place. This way when you launch an instance from that AMI the new instance can generate the administrator password and provide it to you in the AWS console. Simple…

Here’s how you tell the EC2Config service to do that.  Connect to the instance before you create an AMI, go to Start -> All Programs -> EC2ConfigService Settings, then go to the “Image” tab and select “random” under “Administrator Password” settings as shown below. To save this setting, click apply.

After saving these settings, you can create an AMI from this instance, and on relaunch the administrator password gets reset successfully. No hijinx necessary.