Windows XP/7/8 virtualisation from a .VHD file, Linux VM Host

I devised the below method/script to create a cloned VM from a running Windows machine.  A VHD snapshot is created  automatically each evening, which can be converted into a VM via the below process.

Target machine drives & partitions, output from diskpart command:

DISKPART> list volume

  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
  ----------  ---  -----------  -----  ----------  -------  ---------  --------
  Volume 0     D                       DVD-ROM         0 B  No Media
  Volume 1         System Rese  NTFS   Partition    100 MB  Healthy    System
  Volume 2     C                NTFS   Partition    931 GB  Healthy    Boot
  Volume 3     E   Backup       NTFS   Partition    298 GB  Healthy

The E:\ drive is not backed up in the below procedure (space restrictions), this does make the process slightly more complex as we can’t use the “*” option to include all volumes with the disk2vhd.exe utility.  We need to attach the above ‘Volume 1′ partition via the diskpart command prior to running the disk2vhd.exe command.  This is done via the diskpart command taking the below diskpart_start.txt file as its input when mapping the M:\ drive.

$ cat diskpart_start.txt
sel disk 0
sel part 1
assign letter=m noerr

The script produces a .VHD file at the below location, the ‘WindowsImageBackup’ folder will have the latest copy:

E:\SystemBackupsPleaseLeave\WindowsImageBackup

Old ‘WindowsImageBackup’ folders are date stamped and rotated weekly.

Script details:

E:
cd E:\SystemBackupsPleaseLeave
REM Delete backups older than 7 days, E:\ drive too small to accommodate more.
forfiles -p E:\SystemBackupsPleaseLeave\ -m WindowsImageBackup* -d -7 -c "cmd /c rmdir /s /q @path"
REM Rename old E:\SystemBackupsPleaseLeave\WindowsImageBackup folder if it exists
rename WindowsImageBackup "WindowsImageBackup%date:/=%-%time::=%RENAMED"
REM Create E:\SystemBackupsPleaseLeave\WindowsImageBackup for each days backup
mkdir E:\SystemBackupsPleaseLeave\WindowsImageBackup
REM Map partition that doesn't have a drive letter (M)
diskpart /s C:\cygwin\usr\local\bin\diskpart_start.txt
REM Pause for 10 seconds
ping -n 10 -w 1000 127.0.0.1 > nul
REM start....
C:\cygwin\usr\local\bin\disk2vhd.exe -accepteula M: C: E:\SystemBackupsPleaseLeave\WindowsImageBackup\snapshot.vhd
REM Pause for 30 seconds
ping -n 10 -w 3000 127.0.0.1 > nul
REM Unmap the partition that doesn't have a drive letter (M)
diskpart /s C:\cygwin\usr\local\bin\diskpart_end.txt

Converting .VHD to VM

I reverted to the below method to work around a dated version of qemu-img on my Debian host, which wouldn’t allow me to convert the .VHD file to a RAW file.  I did explore using VBoxManage on my workstation but it too didn’t manage to complete the conversion.

Install cygwin sshd onto another Windows 7 machine, start sshd. This machine will be used to mount the .VHD file and to make it available for copying:

Mount the .VHD file as a read only drive via the disk management utility.

The drive will now be accessible under the cygwin /dev directory. For example /dev/sdb.

From your linux VM host run the below command to copy the above drive (all partitions) as a sparse file.  In the below command the Windows 7 machine with the mounted VHD has an IP of 192.168.1.12

ssh username@192.168.1.12 'dd if=/dev/sdb | gzip ' | gunzip | cp --sparse=always /proc/self/fd/0 /data/vm/clone/windows_machine_sparse_file

source: http://unix.stackexchange.com/questions/25528/is-creating-a-sparse-image-using-dd-appropriate-for-backup-restore-from-a-raid

Attach the /data/vm/clone/windows_machine_sparse file to a VM on your Linux host, remember to also attach the Windows DVD.

The sequence/procedure from here down will vary slightly, I guess, if the original Windows workstation is not Windows 7.

Launching this VM (view using VNC) will result in the below error:

Error

After selecting the DVD the boot process will commence:

Booting

Live

Live

Select repair your computer.

Live

Live

Choose ‘Command prompt’ on the below screen

Live

Live

Now use the DISKPART utility to select the machine’s C:\ drive, using the below commands:

source: http://www.sevenforums.com/installation-setup/88872-startup-repair-menu-doesnt-see-operating-system.html , see gregrocker post:

DISKPART
LIST DISK
SELECT DISK 0 (confirm from list this is Windows 7 HD #)
LIST PARTITION
SELECT PARTITION # (replace # with Windows 7 part #)
ACTIVE
EXIT

Live

I’m not 100% sure if the below steps are necessary, nothing lost doing them, enter the below commands into the same DOS prompt.

source:

http://answers.microsoft.com/en-us/windows/forum/windows_7-system/corrupt-file-at-startup-can-not-start-up-win-7/b7129bd4-abf5-44e8-b136-9d9bb9a22c60

bootrec.exe /fixmbr
bootrec.exe /fixboot
bootrec.exe /RebuildBcd

Live

With the drive mounted, it should be accessible via the same DOS prompt:

Live

Changing the Registry

The below steps will alter the registry of the machine to allow it to boot as a VM.

Run the below commands in the same DOS prompt to load the registry from the D:\ drive, this might differ for you.

reg load HKLM\VHDSYS D:\windows\system32\config\system
regedit

Live

I’m not 100% sure if both CurrentControlSet0001 & CurrentControlSet0002 need to be changed, won’t hurt.

Navigate to:

HKLM/VHDSYS/CurrentControlSet0001/Services/

&

HKLM/VHDSYS/CurrentControlSet0002/Services/

Change the value of the “Start” parameter under the below keys.  Only values with stars need changing on a KVM Linux virtual machine, check others.

Aliide = 3
Amdide = 3
Atapi = 0
Cmdide = 3
iaStorV = 3
*intelide = 0
*msahci = 3
*pciide = 3
viaide = 3

Close regedit once you have finished the above edits.

Run the below command to unload the mounted registry from memory.

reg unload HKLM\VHDSYS

Reboot the machine after these changes, using the ‘Restart’ button:

Live

Choose the DVD as the boot device again:

Live

Live

Choose the ‘Repair and restart’ option.

Live

Choose the DVD as the boot device again:

Live

Live

Live

Live

Now attempt to boot from the hard drive:

Live

Live

Live

Live

Hopefully voila!

Allow Smartmontools to send emails via Gmail, using exim

Allow Smartmontools to relay mail via Gmail utilising the robust lightweight Exim mail transfer agent.

Install exim:

apt-get install exim4-base exim4-config
apt-get install exim4

Exim configuration:

1-smarthost

2-localhost

3-127.0.0.1

4-Other_destinations

5-relay_for

6-smtp.gmail.comb

7-Hide_local_mail_name_in_outgoing_mail

8-Keep_number_of_DNS-queries_minimal

9-mbox_format_in

10-split_config

Edit your credentials:

john@mylocalmachine:/home/john# cat /etc/exim4/passwd.client
# password file used when the local exim is authenticating to a remote
# host as a client.
#
# see exim4_passwd_client(5) for more documentation
#
# Example:
### target.mail.server.example:login:password
*.google.com:youraddress@domain.com:YOURPASSWORD

Secure your credentials:

chmod 640 /etc/exim4/passwd.client

This two step process ensures Exim4’s configuration is updated and the deamon is restarted:

update-exim4.conf
invoke-rc.d exim4 restart

Benign error starting exim, on my system:

ALERT: exim paniclog /var/log/exim4/paniclog has non-zero size, mail system possibly broken
rm /var/log/exim4/paniclog

Confirm that Exim is operational via:

echo testing | mail -s Bla myemail@somewhere.com

Checking that smart monitoring is enabled at boot (on Debian based systems):

sysv-rc-conf

Sysv

Installing the below mailutils will allow the smartd daemon to send emails via exim (configured for gmail relay) :

apt-get install mailutils

Adjust your Smartmontools configuration so it sends out an email on start up and when a problem is encountered with one of your disks.

# cat /etc/smartd.conf
/dev/sg1 -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m address@domain.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sg2 -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m address@domain.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sg3 -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m address@domain.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sg4 -a -d sat -o on -S on -s (S/../.././02|L/../../6/03) -m address@domain.com -M exec /usr/share/smartmontools/smartd-runner
/dev/sg4 -H -m address@domain.com -M test

You will need the below line to send out a test email on smartd start up.

/dev/sg4 -H -m address@domain.com -M test

Virtual Windows 8 (x86/x64) under Linux KVM/QEMU

I’ve spent way too much time getting Windows 8 to play ball under KVM/QEMU on my Debian host.

The first hurdle was to emulate the CPU prerequisites (Physical Address Extension (PAE), NX, and SSE2).  This was accomplished with the below CPU deceleration in the machine’s XML file.

<cpu mode='custom' match='minimum'>
 <model fallback='allow'>core2duo</model>
 <feature policy='require' name='lahf_lm'/>
 <feature policy='require' name='sse2'/>
 <feature policy='require' name='xtpr'/>
 <feature policy='require' name='cx16'/>
 <feature policy='require' name='tm2'/>
 <feature policy='require' name='est'/>
 <feature policy='require' name='vmx'/>
 <feature policy='require' name='ds_cpl'/>
 <feature policy='require' name='pbe'/>
 <feature policy='require' name='tm'/>
 <feature policy='require' name='ht'/>
 <feature policy='require' name='ss'/>
 <feature policy='require' name='acpi'/>
 <feature policy='require' name='ds'/>
 <feature policy='require' name='nx'/>
 <feature policy='require' name='pae'/>
 <feature policy='require' name='mmx'/>
 </cpu>

Thanks to VMcPherron for posting this information on http://superuser.com

An almost working copy of the XML looks like the below:

<domain type='kvm' id='168'>
 <name>Virtual8</name>
 <uuid>1a86fde6-45b3-bd10-9d96-9c17de4f2a9e</uuid>
 <memory unit='KiB'>2097152</memory>
 <currentMemory unit='KiB'>2097152</currentMemory>
 <vcpu>2</vcpu>
 <os>
 <type arch='x86_64' machine='pc-0.14'>hvm</type>
 <boot dev='cdrom'/>
 <boot dev='hd'/>
 <bootmenu enable='yes'/>
 </os>
 <features>
 <acpi/>
 <apic/>
 <pae/>
 </features>
 <cpu mode='custom' match='minimum'>
 <model fallback='allow'>core2duo</model>
 <feature policy='require' name='lahf_lm'/>
 <feature policy='require' name='sse2'/>
 <feature policy='require' name='xtpr'/>
 <feature policy='require' name='cx16'/>
 <feature policy='require' name='tm2'/>
 <feature policy='require' name='est'/>
 <feature policy='require' name='vmx'/>
 <feature policy='require' name='ds_cpl'/>
 <feature policy='require' name='pbe'/>
 <feature policy='require' name='tm'/>
 <feature policy='require' name='ht'/>
 <feature policy='require' name='ss'/>
 <feature policy='require' name='acpi'/>
 <feature policy='require' name='ds'/>
 <feature policy='require' name='nx'/>
 <feature policy='require' name='pae'/>
 <feature policy='require' name='mmx'/>
 </cpu>
 <clock offset='localtime'/>
 <on_poweroff>destroy</on_poweroff>
 <on_reboot>restart</on_reboot>
 <on_crash>destroy</on_crash>
 <devices>
 <emulator>/usr/bin/kvm</emulator>
 <disk type='file' device='disk'>
 <driver name='qemu' type='qcow2'/>
 <source file='/vm/clone/Virtual8.qcow2'/>
 <target dev='hda' bus='ide'/>
 <alias name='ide0-0-0'/>
 <address type='drive' controller='0' bus='0' target='0' unit='0'/>
 </disk>
 <disk type='file' device='cdrom'>
 <driver name='qemu' type='raw'/>
 <source file='/data/isos/en-gb_windows_8_enterprise_x64_dvd_922086.iso'/>
 <target dev='hdc' bus='ide'/>
 <readonly/>
 <alias name='ide0-1-0'/>
 <address type='drive' controller='0' bus='1' target='0' unit='0'/>
 </disk>
 <controller type='ide' index='0'>
 <alias name='ide0'/>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
 </controller>
 <controller type='usb' index='0'>
 <alias name='usb0'/>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
 </controller>
 <interface type='bridge'>
 <mac address='00:16:3a:d2:eb:27'/>
 <source bridge='br0'/>
 <target dev='vnet15'/>
 <alias name='net0'/>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
 </interface>
 <serial type='pty'>
 <source path='/dev/pts/19'/>
 <target port='0'/>
 <alias name='serial0'/>
 </serial>
 <console type='pty' tty='/dev/pts/19'>
 <source path='/dev/pts/19'/>
 <target type='serial' port='0'/>
 <alias name='serial0'/>
 </console>
 <input type='tablet' bus='usb'>
 <alias name='input0'/>
 </input>
 <input type='mouse' bus='ps2'/>
 <graphics type='vnc' port='5915' autoport='yes'/>
 <video>
 <model type='xen' vram='18432' heads='1'/>
 <alias name='video0'/>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
 </video>
 <memballoon model='virtio'>
 <alias name='balloon0'/>
 <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
 </memballoon>
 </devices>
 <seclabel type='none'/>
</domain>

In the above code you will need to replace:

<model type='xen' vram='18432' heads='1'/>

with:

<model type='vga' vram='18432' heads='1'/>

for the install.

On my system (/usr/bin/kvm: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, stripped) I found the VM crashed after booting from a shutdown, it was fine between reboots though?

It’s best not to shutdown the VM until you have finalised the install, then make the change from

type='vga'

to

type='xen'

It will crash on first boot, but will be fine for subsequent shut-downs and reboots.

The main hurdle I’m still facing is not being able to VNC into the machine (black/blank screen) with the above set-up, RDP works fine though. I’m able to VNC into the VM with the problematic video card settings:

type='vga'

but not:

type='xen'

I’ve lowered the graphics requirements by choosing ‘Adjust for best performance’ within Windows 8 but it doesn’t make a difference.

Windows 2012 server will install and run fine with the above XML, keep the video as:

type='vga'

Seems like Windows 8 is a bit more neurotic about video cards :-(

I would appreciate your feedback on getting Windows 8 running both RDP and VNC under Linux/KVM/QEMU.