XenServer will create sparse disks for your VMs only on NFS based volumes. If you are using ISCSI or local storage, it will fully allocate each disk. This is less of a problem when you have a thinly provisioned ISCSI volume which is also compressed (such as those provided by Nexenta/OpenSolaris), but when you have a limited amount of local storage, it's unfortunate that XenServer doesn't give you the option to create sparse disks.

Luckily, XenServer is just CentOS 5 under the hood, and we can turn the local storage into an NFS volume.

First we will need to delete the current Local Storage, which involves quite a few steps.

view plain print about
1#lvdisplay
2 --- Logical volume ---
3 LV Name /dev/VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2/MGT
4 VG Name VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
5 LV UUID HlifV3-7B6J-rVCp-pAxP-Rc7e-viEQ-Tt819a
6 LV Write Access read/write
7 LV Status NOT available
8 LV Size 4.00 MB
9 Current LE 1
10 Segments 1
11 Allocation inherit
12 Read ahead sectors auto
Now we run xe pbd-list and find the one that matches our local SR.
view plain print about
1#xe pbd-list
2uuid ( RO) : e06a37a5-e1e6-4415-134e-f5ae4f7f7c17
3 host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
4 sr-uuid ( RO): fc8c8fc5-4a06-18b9-517d-f19a18e50820
5 device-config (MRO): location: /dev/xapi/cd
6 currently-attached ( RO): true
7
8
9uuid ( RO) : 7a75d1b9-9aa5-d5f7-6b85-50d1175054c8
10 host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
11 sr-uuid ( RO): a6db482a-5446-3fc3-6513-6933eb9d7915
12 device-config (MRO): location: /dev/xapi/block
13 currently-attached ( RO): true
14
15
16uuid ( RO) : 6c1eeeb0-3c05-fbee-4dca-e7490e007504
17 host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
18 sr-uuid ( RO): f663b0ec-56b0-72dc-d546-4e0fc9be7ef6
19 device-config (MRO): location: /opt/xensource/packages/iso; legacy_mode: true
20 currently-attached ( RO): true
21uuid ( RO) : 825bc0eb-8ec9-01ab-e249-21146d66dd9a
22 host-uuid ( RO): 99ae2c2e-b992-4b4d-a09d-15cba5483225
23 sr-uuid ( RO): 877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
24 device-config (MRO): device: /dev/disk/by-id/scsi-SATA_WDC_WD800JD-75M_WD-WMAM9AJ38489-part3
25 currently-attached ( RO): true

As we can see, it's the last one. Now we unplug it and destroy it.

view plain print about
1#xe pbd-unplug uuid=825bc0eb-8ec9-01ab-e249-21146d66dd9a
2#xe pbd-destroy uuid=825bc0eb-8ec9-01ab-e249-21146d66dd9a

Now lets destroy the storage repository attached to it. First we get a list of the repositories.

view plain print about
1#xe sr-list
2uuid ( RO) : a6db482a-5446-3fc3-6513-6933eb9d7915
3 name-label ( RW): Removable storage
4 name-description ( RW):
5 host ( RO): xenserver-ueyqfddq
6 type ( RO): udev
7 content-type ( RO): disk
8
9
10uuid ( RO) : fc8c8fc5-4a06-18b9-517d-f19a18e50820
11 name-label ( RW): DVD drives
12 name-description ( RW): Physical DVD drives
13 host ( RO): xenserver-ueyqfddq
14 type ( RO): udev
15 content-type ( RO): iso
16
17
18uuid ( RO) : 877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
19 name-label ( RW): Local storage
20 name-description ( RW):
21 host ( RO): <not in database>
22 type ( RO): lvm
23 content-type ( RO): user
24
25
26uuid ( RO) : f663b0ec-56b0-72dc-d546-4e0fc9be7ef6
27 name-label ( RW): XenServer Tools
28 name-description ( RW): XenServer Tools ISOs
29 host ( RO): xenserver-ueyqfddq
30 type ( RO): iso
31 content-type ( RO): iso
We will delete the one called local storage.

view plain print about
1#xe sr-forget uuid=877a1f66-59dd-b1ba-0de3-d8a753d0a0b2

At this point it will be forgotten and disappear from XenCenter.

Now, lets delete it from LVM.

view plain print about
1#vgdisplay
2 --- Volume group ---
3 VG Name VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
4 System ID
5 Format lvm2
6 Metadata Areas 1
7 Metadata Sequence No 3
8 VG Access read/write
9 VG Status resizable
10 MAX LV 0
11 Cur LV 1
12 Open LV 0
13 Max PV 0
14 Cur PV 1
15 Act PV 1
16 VG Size 66.85 GB
17 PE Size 4.00 MB
18 Total PE 17113
19 Alloc PE / Size 1 / 4.00 MB
20 Free PE / Size 17112 / 66.84 GB
21 VG UUID 2PJJkR-ULpa-1F6f-8H65-N22A-o11C-f0KkGa

Now lets remove it.

view plain print about
1#vgremove VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2
2Do you really want to remove volume group "VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2" containing 1 logical volumes? [y/n]: y
3 Logical volume "MGT" successfully removed
4 Volume group "VG_XenStorage-877a1f66-59dd-b1ba-0de3-d8a753d0a0b2" successfully removed

view plain print about
1#pvdisplay
2 "/dev/sda3" is a new physical volume of "66.86 GB"
3 --- NEW Physical volume ---
4 PV Name /dev/sda3
5 VG Name
6 PV Size 66.86 GB
7 Allocatable NO
8 PE Size (KByte) 0
9 Total PE 0
10 Free PE 0
11 Allocated PE 0
12 PV UUID ZK23c0-UvDg-A4MB-TWKi-YfNW-hWUK-uoyqdo
Now we remove the physical volume from LVM.
view plain print about
1#pvremove /dev/sda3
2 Labels on physical volume "/dev/sda3" successfully wiped

Now lets create a filesystem for us to use.

view plain print about
1mkfs.ext3 -m 0 /dev/sda3
2mke2fs 1.39 (29-May-2006)
3Filesystem label=
4OS type: Linux
5Block size=4096 (log=2)
6Fragment size=4096 (log=2)
78765440 inodes, 17526915 blocks
80 blocks (0.00%) reserved for the super user
9First data block=0
10Maximum filesystem blocks=0
11535 block groups
1232768 blocks per group, 32768 fragments per group
1316384 inodes per group
14Superblock backups stored on blocks:
1532768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
164096000, 7962624, 11239424
17
18Writing inode tables: done
19Creating journal (32768 blocks): done
20Writing superblocks and filesystem accounting information: done
21
22This filesystem will be automatically checked every 33 mounts or
23180 days, whichever comes first. Use tune2fs -c or -i to override.

Now lets set up a folder for it and make sure it gets mounted at boot time.

view plain print about
1#mkdir /data

view plain print about
1#vi /etc/fstab
2LABEL=root-cjfffcbq /ext3 defaults 1 1
3/var/swap/swap.001swap swap defaults 0 0
4none/dev/pts devpts defaults 0 0
5none/dev/shm tmpfs defaults 0 0
6none/proc proc defaults 0 0
7none/sys sysfs defaults 0 0

We need to add our directory to automatically mount at boot time. Lets add this entry

view plain print about
1/dev/sda3 /data ext3 defaults 1 2

This is how my file looked like at the end.

view plain print about
1/ext3 defaults 1 1
2/var/swap/swap.001swap swap defaults 0 0
3none/dev/pts devpts defaults 0 0
4none/dev/shm tmpfs defaults 0 0
5none/proc proc defaults 0 0
6none/sys sysfs defaults 0 0
7/dev/sda3 /data ext3 defaults 1 2

Now lets try mounting it.

view plain print about
1#mount /data

If you get no output, that means it probably worked.

Lets check that it mounted successfully.

view plain print about
1#ls /data
2lost+found

Now lets create our directories that we will export.

view plain print about
1#mkdir /data/vm
2#mkdir /data/iso

Lets edit the exports file.

view plain print about
1#vi /etc/exports
The file should initially be empty. Lets add our entries.
view plain print about
1/data/vm 127.0.0.1(rw,no_root_squash,sync)
2/data/iso 127.0.0.1(ro,no_root_squash,sync)

Save the file, and now lets start the nfs and portmap services.

view plain print about
1#service nfs start
2Starting NFS services: [ OK ]
3Starting NFS daemon: [ OK ]
4Starting NFS mountd: [ OK ]
5#service portmap start
6Starting portmap: [ OK ]

Lets make sure that the portmap and nfs services start at boot.

view plain print about
1#chkconfig --level 345 nfs on
2#chkconfig --level 345 portmap on

Lets verify that our directories are being exported.

view plain print about
1#exportfs
2/data/iso 127.0.0.1
3/data/vm 127.0.0.1

Now we can add the Storage Repository. We can do it through XenCenter. Right click on the server in XenCenter, and click "New Storage Repository". Select "NFS VHD" as the type, and enter the following parameters.

view plain print about
1Name: NFS Local Storage
2Share Name: 127.0.0.1:/data/vm
3Leave the checkbox on "Create a New SR" and click "Finish".

Lets add an ISO SR as well. Right click on the server in XenCenter, and click "New Storage Repository". Select "NFS ISO" as the type, and enter the following parameters.

view plain print about
1Name: NFS Local Storage
2Share Name: 127.0.0.1:/data/iso
3Leave the checkbox on "Create a New SR" and click "Finish".

Now we reboot and if everything comes back up, we are done. Now when we create VMs, the Virtual Hard Disks will be allocated sparsely, and VMs created from templates, will all use the same base VHD and only store the changes in their own disk.

This entry was posted on July 30, 2009 at 11:24 AM and has received 13127 views. There are currently 69 comments. Print this entry.