XenServer 5.5 - Converting Local Storage to NFS (and thereby enabling sparse provisioning)

Posted At : July 30, 2009 10:24 AM | Posted By : Ruslan Sivak
Related Categories: XenServer
XenServer will create sparse disks for your VMs only on NFS based volumes. If you are using ISCSI or local storage, it will fully allocate each disk. This is less of a problem when you have a thinly provisioned ISCSI volume which is also compressed (such as those provided by Nexenta/OpenSolaris), but when you have a limited amount of local storage, it's unfortunate that XenServer doesn't give you the option to create sparse disks.

Luckily, XenServer is just CentOS 5 under the hood, and we can turn the local storage into an NFS volume.

First we will need to delete the current Local Storage, which involves quite a few steps.

#lvdisplay
--- Logical volume ---
LV Name /dev/VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2/MGT
VG Name VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
LV UUID HlifV3-7B6J-rVCp-pAxP-Rc7e-viEQ-Tt819a
LV Write Access read/write
LV Status NOT available
LV Size 4.00 MB
Current LE 1
Segments 1
Allocation inherit
Read ahead sectors auto
Now we run xe pbd-list and find the one that matches our local SR.
#xe pbd-list
uuid ( RO) : e06a37a5-e1e6-4415-134e-f5ae4f7f7c17
host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
sr-uuid ( RO): fc8c8fc5-4a06-18b9-517d-f19a18e50820
device-config (MRO): location: /dev/xapi/cd
currently-attached ( RO): true


uuid ( RO) : 7a75d1b9-9aa5-d5f7-6b85-50d1175054c8
host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
sr-uuid ( RO): a6db482a-5446-3fc3-6513-6933eb9d7915
device-config (MRO): location: /dev/xapi/block
currently-attached ( RO): true


uuid ( RO) : 6c1eeeb0-3c05-fbee-4dca-e7490e007504
host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
sr-uuid ( RO): f663b0ec-56b0-72dc-d546-4e0fc9be7ef6
device-config (MRO): location: /opt/xensource/packages/iso; legacy_mode: true
currently-attached ( RO): true
uuid ( RO) : 825bc0eb-8ec9-01ab-e249-21146d66dd9a
host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
sr-uuid ( RO): 877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
device-config (MRO): device: /dev/disk/by-id/scsi-SATA_WDC_WD800JD-75M_WD-WMAM9AJ38489-part3
currently-attached ( RO): true

As we can see, it's the last one. Now we unplug it and destroy it.

#xe pbd-unplug uuid=825bc0eb-8ec9-01ab-e249-21146d66dd9a
#xe pbd-destroy uuid=825bc0eb-8ec9-01ab-e249-21146d66dd9a

Now lets destroy the storage repository attached to it. First we get a list of the repositories.

#xe sr-list
uuid ( RO) : a6db482a-5446-3fc3-6513-6933eb9d7915
name-label ( RW): Removable storage
name-description ( RW):
host ( RO): xenserver-ueyqfddq
type ( RO): udev
content-type ( RO): disk


uuid ( RO) : fc8c8fc5-4a06-18b9-517d-f19a18e50820
name-label ( RW): DVD drives
name-description ( RW): Physical DVD drives
host ( RO): xenserver-ueyqfddq
type ( RO): udev
content-type ( RO): iso


uuid ( RO) : 877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
name-label ( RW): Local storage
name-description ( RW):
host ( RO): <not in database>
type ( RO): lvm
content-type ( RO): user


uuid ( RO) : f663b0ec-56b0-72dc-d546-4e0fc9be7ef6
name-label ( RW): XenServer Tools
name-description ( RW): XenServer Tools ISOs
host ( RO): xenserver-ueyqfddq
type ( RO): iso
content-type ( RO): iso
We will delete the one called local storage.

#xe sr-forget uuid=877a1f66-59dd-b1ba-0de3-d8a753d0a0b2

At this point it will be forgotten and disappear from XenCenter.

Now, lets delete it from LVM.

#vgdisplay
--- Volume group ---
VG Name VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 3
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 0
Max PV 0
Cur PV 1
Act PV 1
VG Size 66.85 GB
PE Size 4.00 MB
Total PE 17113
Alloc PE / Size 1 / 4.00 MB
Free PE / Size 17112 / 66.84 GB
VG UUID 2PJJkR-ULpa-1F6f-8H65-N22A-o11C-f0KkGa

Now lets remove it.

#vgremove VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
Do you really want to remove volume group "VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2" containing 1 logical volumes? [y/n]: y
Logical volume "MGT" successfully removed
Volume group "VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2" successfully removed

#pvdisplay
"/dev/sda3" is a new physical volume of "66.86 GB"
--- NEW Physical volume ---
PV Name /dev/sda3
VG Name
PV Size 66.86 GB
Allocatable NO
PE Size (KByte) 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID ZK23c0-UvDg-A4MB-TWKi-YfNW-hWUK-uoyqdo
Now we remove the physical volume from LVM.
#pvremove /dev/sda3
Labels on physical volume "/dev/sda3" successfully wiped

Now lets create a filesystem for us to use.

mkfs.ext3 -m 0 /dev/sda3
mke2fs 1.39 (29-May-2006)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
8765440 inodes, 17526915 blocks
0 blocks (0.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=0
535 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424

Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 33 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Now lets set up a folder for it and make sure it gets mounted at boot time.

#mkdir /data

#vi /etc/fstab
LABEL=root-cjfffcbq /ext3 defaults 1 1
/var/swap/swap.001swap swap defaults 0 0
none/dev/pts devpts defaults 0 0
none/dev/shm tmpfs defaults 0 0
none/proc proc defaults 0 0
none/sys sysfs defaults 0 0

We need to add our directory to automatically mount at boot time. Lets add this entry

/dev/sda3 /data ext3 defaults 1 2

This is how my file looked like at the end.

/ext3 defaults 1 1
/var/swap/swap.001swap swap defaults 0 0
none/dev/pts devpts defaults 0 0
none/dev/shm tmpfs defaults 0 0
none/proc proc defaults 0 0
none/sys sysfs defaults 0 0
/dev/sda3 /data ext3 defaults 1 2

Now lets try mounting it.

#mount /data

If you get no output, that means it probably worked.

Lets check that it mounted successfully.

#ls /data
lost+found

Now lets create our directories that we will export.

#mkdir /data/vm
#mkdir /data/iso

Lets edit the exports file.

#vi /etc/exports
The file should initially be empty. Lets add our entries.
/data/vm 127.0.0.1(rw,no_root_squash,sync)
/data/iso 127.0.0.1(ro,no_root_squash,sync)

Save the file, and now lets start the nfs and portmap services.

#service nfs start
Starting NFS services: [ OK ]
Starting NFS daemon: [ OK ]
Starting NFS mountd: [ OK ]
#service portmap start
Starting portmap: [ OK ]

Lets make sure that the portmap and nfs services start at boot.

#chkconfig --level 345 nfs on
#chkconfig --level 345 portmap on

Lets verify that our directories are being exported.

#exportfs
/data/iso 127.0.0.1
/data/vm 127.0.0.1

Now we can add the Storage Repository. We can do it through XenCenter. Right click on the server in XenCenter, and click "New Storage Repository". Select "NFS VHD" as the type, and enter the following parameters.

Name: NFS Local Storage
Share Name: 127.0.0.1:/data/vm
Leave the checkbox on "Create a New SR" and click "Finish".

Lets add an ISO SR as well. Right click on the server in XenCenter, and click "New Storage Repository". Select "NFS ISO" as the type, and enter the following parameters.

Name: NFS Local Storage
Share Name: 127.0.0.1:/data/iso
Leave the checkbox on "Create a New SR" and click "Finish".

Now we reboot and if everything comes back up, we are done. Now when we create VMs, the Virtual Hard Disks will be allocated sparsely, and VMs created from templates, will all use the same base VHD and only store the changes in their own disk.

Comments (56) | Print | Send | del.icio.us | Linking Blogs

Jumbo Frames with XenServer 5.5

Posted At : July 29, 2009 10:37 PM | Posted By : Ruslan Sivak
Related Categories: XenServer
There doesn't seem to be any way to configure jumbo frames with XenServer 5.5, and I'm sure this is unsupported, but it can be done by hacking the XenServer 5.5 config files.

#vi /opt/xensource/libexec/interface-reconfigure +879

You will see something like this

f.write('"\n')
return f

We are going to add our MTU information there:

f.write('"\n')
#add MTU
f.write('MTU=9212\n');
return f

9212 is the MTU I chose, make sure you choose one that's appropriate for your network, and make sure to test it first.

This will only change the MTU on the bonded interfaces, I believe.

Comments (54) | Print | Send | del.icio.us | Linking Blogs

CentOS 5.3 on XenServer 5.5 Graphical Install

Posted At : July 15, 2009 9:45 PM | Posted By : Ruslan Sivak
Related Categories: Virtualization, XenServer, CentOS
In order to do a graphical install (which lets you customize your LVM volumes) you just need to add the vnc option to the "OS Boot Parameters" when creating the VM. It should look like this:
graphical utf8 vnc

Once you start up your VM, it will go through a short text based configuration wizard, configuring your NIC and such, after which it will prompt you to log in using VNC.

Comments (55) | Print | Send | del.icio.us | Linking Blogs

LACP with XenServer 5.5

Posted At : July 10, 2009 7:27 PM | Posted By : Ruslan Sivak
Related Categories: XenServer
In order to make XenServer 5.5 work with LACP, we need to modify the start up scripts
#vi /opt/xensource/libexec/interface-reconfigure +863

We will see the following:

# The bond option defaults
bond_options = {
"mode": "balance-slb",
"miimon": "100",
"downdelay": "200",
"updelay": "31000",
"use_carrier": "1",
}

We will change the mode from "balance-slb" to "802.3ad"

# The bond option defaults
bond_options = {
"mode": "802.3ad",
"miimon": "100",
"downdelay": "200",
"updelay": "31000",
"use_carrier": "1",
}

Don't forget to configure your switch for LACP.

Comments (36) | Print | Send | del.icio.us | Linking Blogs

Detect new network cards on XenServer 5

Posted At : April 1, 2009 7:20 PM | Posted By : Ruslan Sivak
Related Categories: Xen, XenServer
I installed XenServer 5, and then added some gigabit NICs, but the server wasn't seeing them. Turned out I needed to do the following from the server console:

#xe host-list
uuid ( RO) : 9ec3de63-30fe-4575-b972-1234ed235345
name-label ( RW): xenserver-lfiuljac
name-description ( RO): Default install of XenServer
#
xe pif-scan host-uuid=9ec3de63-30fe-4575-b972-1234ed235345

This made the NICs show up in XenCenter.

Comments (30) | Print | Send | del.icio.us | Linking Blogs
BlogCFC was created by Raymond Camden.