Ruslan Sivak: Xen 3.2 on CentOS 5.2 x86_64 / RHEL 5.2 x86

Xen 3.2 on CentOS 5.2 x86_64 / RHEL 5.2 x86_64

Jul 7

posted by Ruslan Sivak in Xen | 125 Comments

Since my drivers are not signed, you will need to temporarily disable signature checking for yum.

view plain print about

1# vi /etc/yum.conf

Make sure it says gpgcheck=0

Download the file and untar it.

view plain print about

1# tar xvzf xen-3.2.0-CentOS5.2.tar.gz
2xen-3.2.0-CentOS5.2/
3xen-3.2.0-CentOS5.2/xen-3.2.0-0xs.x86_64.rpm
4xen-3.2.0-CentOS5.2/xen-libs-3.2.0-0xs.x86_64.rpm
5xen-3.2.0-CentOS5.2/xen-devel-3.2.0-0xs.x86_64.rpm
6
7# cd xen-3.2.0-CentOS5.2

Install the RPMs.

view plain print about

1#yum install -y xen-3.2.0-0xs.x86_64.rpm xen-libs-3.2.0-0xs.x86_64.rpm xen-devel-3.2.0-0xs.x86_64.rpm

It will download a bunch of dependencies and install them. Among them will be a new kernel - kernel-xen.

view plain print about

1# vi /etc/yum.conf

Set gpgcheck back to the old value: gpgcheck=1 if it was originally 1.

The only problem at this point is that yum for some reason doesn't update the boot entry, so we have to do this manually.

view plain print about

1# vi /boot/grub/menu.lst

view plain print about

1# grub.conf generated by anaconda
2#
3# Note that you do not have to rerun grub after making changes to this file
4# NOTICE: You have a /boot partition. This means that
5# all kernel and initrd paths are relative to /boot/, eg.
6# root (hd0,0)
7# kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
8# initrd /initrd-version.img
9#boot=/dev/sda
10default=1
11timeout=5
12splashimage=(hd0,0)/grub/splash.xpm.gz
13hiddenmenu
14title CentOS (2.6.18-92.1.6.el5xen)
15 root (hd0,0)
16 kernel /xen.gz-2.6.18-92.1.6.el5
17 module /vmlinuz-2.6.18-92.1.6.el5xen ro root=/dev/VolGroup00/LogVol00
18 module /initrd-2.6.18-92.1.6.el5xen.img
19title CentOS (2.6.18-92.el5)
20 root (hd0,0)
21 kernel /vmlinuz-2.6.18-92.el5 ro root=/dev/VolGroup00/LogVol00
22 initrd /initrd-2.6.18-92.el5.img

We want to update the CentOS (2.6.18-53.1.21.el5xen) entry, and set it to use the proper kernel - xen.gz-3.2

This is what my updated menu.lst looks like - yours may differ. I have also set the default to 0, so that it boots my xen kernel automatically.

view plain print about

1# grub.conf generated by anaconda
2#
3# Note that you do not have to rerun grub after making changes to this file
4# NOTICE: You have a /boot partition. This means that
5# all kernel and initrd paths are relative to /boot/, eg.
6# root (hd0,0)
7# kernel /vmlinuz-version ro root=/dev/VolGroup00/LogVol00
8# initrd /initrd-version.img
9#boot=/dev/sda
10default=0
11timeout=5
12splashimage=(hd0,0)/grub/splash.xpm.gz
13hiddenmenu
14title CentOS (2.6.18-92.1.6.el5xen)
15 root (hd0,0)
16 kernel /xen.gz-3.2
17 module /vmlinuz-2.6.18-92.1.6.el5xen ro root=/dev/VolGroup00/LogVol00
18 module /initrd-2.6.18-92.1.6.el5xen.img
19title CentOS (2.6.18-92.el5)
20 root (hd0,0)
21 kernel /vmlinuz-2.6.18-92.el5 ro root=/dev/VolGroup00/LogVol00
22 initrd /initrd-2.6.18-92.el5.img

Once you reboot you should automatically boot into the xen kernel and be able to create virtual machines.

This entry was posted on July 7, 2008 at 4:02 PM and has received 24479 views. There are currently 125 comments. Print this entry. Download attachment.

Related Blog Entries

3ware 9690SA on CentOS 5.2 x86_64 with Xen (June 30, 2008)
Xen 3.2 on CentOS 5.1 x86_64 / RHEL 5.1 x86_64 (June 9, 2008)

TweetBacks

Comments

[Add Comment] [Subscribe to Comments]

#1 by Mark on 7/10/08 - 3:15 AM

Nice tutorial. Can you create some rpm's for xen 3.2.1? =)
#2 by Brian on 7/15/08 - 6:27 PM

Have you had any luck with the virt-manager tools? After installing 3.2, I can no longer create new virtual machines using those methods.
#3 by Paul on 7/16/08 - 9:35 PM

Brian,
Did you ever get your virt-manager to work after upgrading to 3.2? I am experiencing the same issue.
#4 by Boris on 7/17/08 - 11:42 AM

virt-install works fine.
I can manage Domains via xm , as i did on Solaris Nevada since build 75a
#5 by Sascha on 7/17/08 - 11:57 AM

Hello,

this tutorial is really pretty good, but i have some addtional stuff. Thank you Ruslan!

@Brian + Paul:
This is a known bug and i've fixed that with the "xen-timer-mode.patch"
There is a yum repository with fixed packages on: www.gitco.de/repo/xen3.2" target="_blank">http://www.gitco.de/repo/xen3.2

@Mark:
Yes, you can use the yum repository on: www.gitco.de/repo/xen3.2" target="_blank">http://www.gitco.de/repo/xen3.2.1
I've just built the packages for this release. Please have a look on this:

======================================================================
[root@slab005 /]# cat /etc/yum.repos.d/XEN.repo
[xen]
name=CentOS-$releasever - XEN
baseurl=www.gitco.de/repo/xen3.2" target="_blank">http://www.gitco.de/repo/xen3.2.1
gpgcheck=0
[root@slab005 /]#

[root@slab005 /]# yum install xen xen-libs xen-devel
Loading "fastestmirror" plugin
Loading mirror speeds from cached hostfile
* xen: www.gitco.de
* extras: ftp.plusline.de
* updates: ftp.plusline.de
* base: ftp.plusline.de
* addons: ftp.plusline.de
Setting up Install Process
Parsing package install arguments
Resolving Dependencies
--> Running transaction check
---> Package xen-devel.x86_64 0:3.2.1-0xs set to be updated
---> Package xen-libs.i686 0:3.0.3-64.el5_2.1 set to be updated
---> Package xen-libs.x86_64 0:3.2.1-0xs set to be updated
---> Package xen.x86_64 0:3.2.1-0xs set to be updated
--> Processing Dependency: python-virtinst for package: xen
---> Package xen-devel.i686 0:3.0.3-64.el5_2.1 set to be updated
--> Running transaction check
---> Package python-virtinst.noarch 0:0.300.2-8.el5 set to be updated
--> Processing Dependency: libvirt-python >= 0.2.1 for package: python-virtinst
--> Processing Dependency: virt-viewer for package: python-virtinst
--> Running transaction check
---> Package virt-viewer.x86_64 0:0.0.2-2.el5 set to be updated
--> Processing Dependency: libvirt.so.0()(64bit) for package: virt-viewer
---> Package libvirt-python.x86_64 0:0.3.3-7.el5 set to be updated
--> Running transaction check
---> Package libvirt.x86_64 0:0.3.3-7.el5 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
xen x86_64 3.2.1-0xs xen 4.4 M
xen-devel x86_64 3.2.1-0xs xen 226 k
xen-devel i686 3.0.3-64.el5_2.1 updates 215 k
Installing for dependencies:
libvirt x86_64 0.3.3-7.el5 base 915 k
libvirt-python x86_64 0.3.3-7.el5 base 74 k
python-virtinst noarch 0.300.2-8.el5 base 161 k
virt-viewer x86_64 0.0.2-2.el5 base 25 k
xen-libs i686 3.0.3-64.el5_2.1 updates 141 k
xen-libs x86_64 3.2.1-0xs xen 365 k

Transaction Summary
=============================================================================
Install 9 Package(s)
Update 0 Package(s)
Remove 0 Package(s)

Total download size: 6.4 M
Is this ok [y/N]: y
Downloading Packages:
(1/8): xen-3.2.1-0xs.x86_ 100% |=========================| 4.4 MB 00:07
(2/8): xen-devel-3.0.3-64 100% |=========================| 215 kB 00:00
(3/8): libvirt-python-0.3 100% |=========================| 74 kB 00:00
(4/8): xen-libs-3.2.1-0xs 100% |=========================| 365 kB 00:00
(5/8): libvirt-0.3.3-7.el 100% |=========================| 915 kB 00:01
(6/8): virt-viewer-0.0.2- 100% |=========================| 25 kB 00:00
(7/8): python-virtinst-0. 100% |=========================| 161 kB 00:00
(8/8): xen-devel-3.2.1-0x 100% |=========================| 226 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Installing: xen-libs ######################### [1/9]
Installing: libvirt ######################### [2/9]
ln: creating symbolic link `/etc/libvirt/qemu/networks/autostart/default.xml' to `../default.xml': File exists
Installing: virt-viewer ######################### [3/9]
Installing: libvirt-python ######################### [4/9]
Installing: xen-libs ######################### [5/9]
Installing: python-virtinst ######################### [6/9]
Installing: xen-devel ######################### [7/9]
Installing: xen-devel ######################### [8/9]
Installing: xen ######################### [9/9]

Installed: xen.x86_64 0:3.2.1-0xs xen-devel.x86_64 0:3.2.1-0xs xen-devel.i686 0:3.0.3-64.el5_2.1
Dependency Installed: libvirt.x86_64 0:0.3.3-7.el5 libvirt-python.x86_64 0:0.3.3-7.el5 python-virtinst.noarch 0:0.300.2-8.el5 virt-viewer.x86_64 0:0.0.2-2.el5 xen-libs.i686 0:3.0.3-64.el5_2.1 xen-libs.x86_64 0:3.2.1-0xs
Complete!

[root@slab005 /]# xm dm | head -n5
__ __ _____ ____ _
\ \/ /___ _ __ |___ / |___ \ / |
\ // _ \ '_ \ |_ \ __) | | |
/ \ __/ | | | ___) | / __/ _| |
/_/\_\___|_| |_| |____(_)_____(_)_|
[root@slab005 /]#

==========================================================================

IMPORTANT: I can't ensure the stability of this packages. So feel free if you deploy that packages in a live environment!!!

Thanks!

Cheers Sascha
#6 by Paul on 7/17/08 - 1:39 PM

Hey Man,

Thanks so much for updating me on this information so quickly. Thank you as well for sharing this with the internet world as well. It saved me a lot of time by using your method to update my CentOS with your RPM packages. I really appreciate the response.

Cheers,

Paul
#7 by Daniel on 7/17/08 - 5:15 PM

Hi Ruslan,

Great work! Did you compile the kernel with "PCI-device backend driver" for VT-d support?

Cheers,
Daniel
#8 by aneip on 7/25/08 - 9:51 AM

Why when I trying to install it still show old version?

Installing:
xen i686 3.0.3-64.el5_2.1 updates 1.8 M
xen-devel i686 3.0.3-64.el5_2.1 updates 215 k

any idea??
#9 by Timothy A. Geist on 7/28/08 - 4:26 PM

I was wondering if there is a similar repo such as the one you mentioned but for a 32bit system?

Tim
#10 by Boris on 7/29/08 - 2:15 PM

View Grant's McWilliams blog (it works for 64 and 32 bit system).
Follow his quite understandable instructions on 32-bit system. You 'll be done
in the same way. You don't need any third party repositories.
Start is always the same
# rpm -iv http://bits.xensource.com/oss-xen/release/3.2.0/ce...- 3.2.0-0xs.centos5.src.rpm 2 > /dev/null
# cd /usr/src/redhat/SOURCES ;
wget http://bits.xensource.com/oss-xen/release/3.2.1/xe...
. . . . . . . . .
# cd /usr/src/redhat/RPMS/i386
You'll get i386 rpms under this folder
#11 by Boris on 7/29/08 - 2:26 PM

Grant's McWilliams blog

http://grantmcwilliams.com/index.php?option=com_co...
#12 by Timothy A. Geist on 7/30/08 - 2:28 PM

Hi Boris

I followed it and it worked perfectly.

Thank you.
#13 by Boris on 7/30/08 - 2:48 PM

Ruslan,
Please, fix a typo in my name in two the most recent comments
been done by me.
My name is Boris.
Thanks.
#14 by Sunny on 8/19/08 - 11:57 AM

I have too : " Installing:
xen i686 3.0.3-64.el5_2.1 updates 1.8 M
xen-devel i686 3.0.3-64.el5_2.1 updates 215 k"

Looks like repo http://www.gitco.de/repo/xen3.2.1 wrong...
#15 by Sascha on 8/19/08 - 1:24 PM

Hello Sunny,

the repo has only packages for arch x86_64. I have a running CentOS5.2 (x86_64) and can't found the packages as mentioned:

Take a look:

[root@slab005 ~]# uname -a
Linux slab005.labor.lan 2.6.18-92.1.6.el5xen #1 SMP Wed Jun 25 14:13:10 EDT 2008 x86_64 x86_64 x86_64 GNU/Linux

[root@slab005 ~]# yum list 'xen*'
Loading "fastestmirror" plugin
Loading mirror speeds from cached hostfile
* xen: www.gitco.de
* extras: mirror.silyus.net
* updates: mirror.silyus.net
* base: mirror.silyus.net
* addons: mirror.silyus.net
Installed Packages
xen.x86_64 3.2.1-0xs installed
xen-devel.x86_64 3.2.1-0xs installed
xen-libs.x86_64 3.2.1-0xs installed
Available Packages
xen-devel.i686 3.0.3-64.el5_2.1 updates
xen-devel.i386 3.0.3-64.el5_2.1 updates
xen-libs.i686 3.0.3-64.el5_2.1 updates
xen-libs.i386 3.0.3-64.el5_2.1 updates
[root@slab005 ~]#

Have you running RHEL or CentOS ?

Cheers,
Sascha
#16 by Mark on 8/24/08 - 2:10 PM

How about Xen 3.3 now that its been released as stable? =P
#17 by Simon on 8/25/08 - 8:40 AM

Yeah I'll make sure to check on your repos for a /3.3.0/
#18 by Boris on 8/26/08 - 4:23 AM

For 3.2 following src rpm was needed:-
http://bits.xensource.com/oss-xen/release/3.2.0/ce...
and that's all
As soon as xen-3.3.0-0xs.centos5.src.rpm will be on the Net. I believe it won't be a problem
to upgrade Xen Hypervisor to 3.3 , but kernel will stay the same.
Cloning Xen 3.3 Mercurial Repository and building xen 3.3 from source requires "git" to be installed before running "make world" and takes more time then for 3.2.
However, works fine on CentOS 5.2.
#19 by Sascha on 8/28/08 - 5:03 PM

Hi all,

i've created the XEN-3.3.0 RPMs today and done a quick test.
On a HVM, it was possible to get a VNC session, but within the virt-manager,
no chance. The virt-manager gives "The console is currently unavailable".
Okay, after a short action of investigation, i had to build a new version of libvirt, too.

These are the packages:

libvirt-0.4.4-3.x86_64.rpm
libvirt-devel-0.4.4-3.x86_64.rpm
libvirt-python-0.4.4-3.x86_64.rpm
xen-3.3.0-2.x86_64.rpm
xen-devel-3.3.0-2.x86_64.rpm
xen-libs-3.3.0-2.x86_64.rpm

You can found them on: http://www.gitco.de/repo/xen3.3.0

And this is the repo file as described above-mentioned:

[root@slab005 /]# cat /etc/yum.repos.d/XEN.repo
[xen]
name=CentOS-$releasever - XEN
baseurl=http://www.gitco.de/repo/xen3.3.0
gpgcheck=0
[root@slab005 /]#

Have a lot of fun ...

IMPORTANT: I can't ensure the stability of this packages. So feel free if you deploy that packages in a live environment!!!

Thanks!

Cheers,
Sascha
#20 by Boris on 8/29/08 - 9:56 AM

Xen 3.3 & libvirtlibvirt-0.4.4-3 rpms installed on CentOS 5.2
Solaris 10 HVM DomU and CentOS 5.2 PV DomU
created via virt-install with no problems.
Thank you
#21 by Nathan on 8/29/08 - 9:43 PM

Transaction Check Error:
file /etc/rc.d/init.d/libvirtd from install of libvirt-0.4.4-3 conflicts with file from package libvirt-0.3.3-7.el5
file /etc/sysconfig/libvirtd from install of libvirt-0.4.4-3 conflicts with file from package libvirt-0.3.3-7.el5
file /usr/share/man/man1/virsh.1.gz from install of libvirt-0.4.4-3 conflicts with file from package
libvirt-0.3.3-7.el5
#22 by boris on 8/30/08 - 3:18 AM

[root@ServerRHL51 InstallXen3.3]# yum install xen-3.3.0-2.x86_64.rpm \
> xen-devel-3.3.0-2.x86_64.rpm xen-libs-3.3.0-2.x86_64.rpm \
> libvirt-0.4.4-3.x86_64.rpm libvirt-devel-0.4.4-3.x86_64.rpm \
> libvirt-python-0.4.4-3.x86_64.rpm
Loading "installonlyn" plugin
Setting up Install Process
Setting up repositories
xen 100% |=========================| 951 B 00:00
Reading repository metadata in from local files
primary.xml.gz 100% |=========================| 3.7 kB 00:00
################################################## 7/7
Parsing package install arguments
Examining xen-3.3.0-2.x86_64.rpm: xen - 3.3.0-2.x86_64
Marking xen-3.3.0-2.x86_64.rpm as an update to xen - 3.2.1-0xs.x86_64
Examining xen-devel-3.3.0-2.x86_64.rpm: xen-devel - 3.3.0-2.x86_64
Marking xen-devel-3.3.0-2.x86_64.rpm as an update to xen-devel - 3.2.1-0xs.x86_64
Examining xen-libs-3.3.0-2.x86_64.rpm: xen-libs - 3.3.0-2.x86_64
Marking xen-libs-3.3.0-2.x86_64.rpm as an update to xen-libs - 3.2.1-0xs.x86_64
Examining libvirt-0.4.4-3.x86_64.rpm: libvirt - 0.4.4-3.x86_64
Marking libvirt-0.4.4-3.x86_64.rpm as an update to libvirt - 0.3.3-7.el5.x86_64
Examining libvirt-devel-0.4.4-3.x86_64.rpm: libvirt-devel - 0.4.4-3.x86_64
Marking libvirt-devel-0.4.4-3.x86_64.rpm to be installed
Examining libvirt-python-0.4.4-3.x86_64.rpm: libvirt-python - 0.4.4-3.x86_64
Marking libvirt-python-0.4.4-3.x86_64.rpm as an update to libvirt-python - 0.3.3-7.el5.x86_64
Resolving Dependencies
--> Populating transaction set with selected packages. Please wait.
---> Package xen.x86_64 0:3.3.0-2 set to be updated
---> Package xen-devel.x86_64 0:3.3.0-2 set to be updated
---> Package xen-libs.x86_64 0:3.3.0-2 set to be updated
---> Package libvirt.x86_64 0:0.4.4-3 set to be updated
---> Package libvirt-devel.x86_64 0:0.4.4-3 set to be updated
---> Package libvirt-python.x86_64 0:0.4.4-3 set to be updated
--> Running transaction check
--> Processing Dependency: /usr/bin/qemu-img for package: libvirt
--> Processing Dependency: iscsi-initiator-utils for package: libvirt
--> Processing Dependency: cyrus-sasl-md5 for package: libvirt
--> Restarting Dependency Resolution with new changes.
--> Populating transaction set with selected packages. Please wait.
---> Downloading header for cyrus-sasl-md5 to pack into transaction set.
cyrus-sasl-md5-2.1.22-4.x 100% |=========================| 15 kB 00:00
---> Package cyrus-sasl-md5.x86_64 0:2.1.22-4 set to be updated
---> Downloading header for qemu to pack into transaction set.
qemu-0.9.0-4.x86_64.rpm 100% |=========================| 13 kB 00:00
---> Package qemu.x86_64 0:0.9.0-4 set to be updated
---> Downloading header for iscsi-initiator-utils to pack into transaction set.
iscsi-initiator-utils-6.2 100% |=========================| 12 kB 00:00
---> Package iscsi-initiator-utils.x86_64 0:6.2.0.868-0.7.el5 set to be updated
--> Running transaction check

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Installing:
libvirt-devel x86_64 0.4.4-3 libvirt-devel-0.4.4-3.x86_64.rpm 1.0 M
Updating:
libvirt x86_64 0.4.4-3 libvirt-0.4.4-3.x86_64.rpm 6.0 M
libvirt-python x86_64 0.4.4-3 libvirt-python-0.4.4-3.x86_64.rpm 339 k
xen x86_64 3.3.0-2 xen-3.3.0-2.x86_64.rpm 15 M
xen-devel x86_64 3.3.0-2 xen-devel-3.3.0-2.x86_64.rpm 843 k
xen-libs x86_64 3.3.0-2 xen-libs-3.3.0-2.x86_64.rpm 1.0 M
Installing for dependencies:
cyrus-sasl-md5 x86_64 2.1.22-4 base 46 k
iscsi-initiator-utils x86_64 6.2.0.868-0.7.el5 base 575 k
qemu x86_64 0.9.0-4 extras 4.5 M

Transaction Summary
=============================================================================
Install 4 Package(s)
Update 5 Package(s)
Remove 0 Package(s)

Total download size: 29 M
Is this ok [y/N]: y
Downloading Packages:
(1/3): cyrus-sasl-md5-2.1 100% |=========================| 46 kB 00:00
(2/3): qemu-0.9.0-4.x86_6 100% |=========================| 4.5 MB 02:35
(3/3): iscsi-initiator-ut 100% |=========================| 575 kB 00:19
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : xen-libs ####################### [ 1/14]
Installing: iscsi-initiator-utils ####################### [ 2/14]
Installing: qemu ####################### [ 3/14]
Installing: cyrus-sasl-md5 ####################### [ 4/14]
Updating : libvirt ####################### [ 5/14]
Updating : libvirt-python ####################### [ 6/14]
Updating : xen ####################### [ 7/14]
Updating : xen-devel ####################### [ 8/14]
Installing: libvirt-devel ####################### [ 9/14]
Cleanup : xen ####################### [10/14]
Cleanup : xen-devel ####################### [11/14]
Cleanup : xen-libs ####################### [12/14]
Cleanup : libvirt ####################### [13/14]
Cleanup : libvirt-python ####################### [14/14]

Installed: libvirt-devel.x86_64 0:0.4.4-3
Dependency Installed: cyrus-sasl-md5.x86_64 0:2.1.22-4 iscsi-initiator-utils.x86_64 0:6.2.0.868-0.7.el5 qemu.x86_64 0:0.9.0-4
Updated: libvirt.x86_64 0:0.4.4-3 libvirt-python.x86_64 0:0.4.4-3 xen.x86_64 0:3.3.0-2 xen-devel.x86_64 0:3.3.0-2 xen-libs.x86_64 0:3.3.0-2
Complete!
[root@ServerRHL51 InstallXen3.3]#
#23 by Sascha on 8/30/08 - 5:36 AM

Hi,

@Nathan:
------------
I think you have installed libvirt.i386, too. For this package, i haven't got built a new RPM.

Take a look on this:

[root@slab005 /]# yum list 'libvirt*'
Loading "fastestmirror" plugin
Loading mirror speeds from cached hostfile
* xen: www.gitco.de
* extras: mirror.fraunhofer.de
* updates: mirror.fraunhofer.de
* base: mirror.fraunhofer.de
* addons: mirror.fraunhofer.de
xen 100% |=========================| 951 B 00:00
extras 100% |=========================| 1.1 kB 00:00
updates 100% |=========================| 951 B 00:00
base 100% |=========================| 1.1 kB 00:00
addons 100% |=========================| 951 B 00:00
Installed Packages
libvirt.x86_64 0.4.4-3 installed
libvirt-python.x86_64 0.4.4-3 installed
Available Packages
libvirt.i386 0.3.3-7.el5 base
libvirt-devel.x86_64 0.4.4-3 xen
libvirt-devel.i386 0.3.3-7.el5 base
[root@slab005 /]#

I only installed libvirt.x86_64 and libvirt-python.x86_64 and not the .i386 version!

@Boris:
---------
Thank you for your comments. I checked this for HVM on Windows Server 2003(x32)/2008(x64) and on WinXP SP3 (x32). And it works, too! A colleague had check this with the newest OpenSolaris (x64) as PVM. No problem at all!

Cheers,
Sascha
#24 by Boris on 8/30/08 - 6:03 AM

I didn't mention Solaris Nevada (b94) PV DomU install only because it was done via
phyton profile referencing "pygrub" as bootloader ( not virt-install)
I also have to notice, that several time graphical console didn't pop up itself after guest's
OS reboot and i had to "vncviewer localhost:0" from other terminal session.
But, it's a minor issue.
#25 by Mickaël on 9/4/08 - 10:25 AM

Hi,

It appears to lack pv-grub.gz kernel in Xen 3.3.0 RPMs provided in http://www.gitco.de/repo/xen3.3.0

Is it normal ? How can I have it ? I would like to test this new feature.

THanks a lot
#26 by Sascha on 9/4/08 - 12:11 PM

Hello Mickaël,

i'll build new RPMs with stubdom on the next days.

Regards,
Sascha
#27 by Mark on 9/6/08 - 5:58 PM

Does anyone have Xen 3.3 and Virt-Manager working fine on CentOS 5.2 x86_64. I basically cant get Virt manager to do much at all anymore =/. Especially create new domu's.
#28 by Boris on 9/9/08 - 3:04 PM

Virt-install works fine.
# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 5946 2 r----- 1004.7
RHEL52PV 2048 1 146.6
# xm start RHEL52PV
# vncviewer localhost:0
#29 by Mark on 9/9/08 - 3:19 PM

Um, xm commands arent the same as using virt-manager (gui app) for managing xen domu's.
#30 by Sascha on 9/9/08 - 3:45 PM

Hi all,

i've created new rpms with stubdom support.

[root@slab004 /]# rpm -q xen
xen-3.3.0-3
[root@slab004 /]# rpm -ql xen |egrep 'stub|pv'
/etc/xen/xmexample.hvm-stubdom
/etc/xen/xmexample.pv-grub
/usr/lib/xen/bin/stubdom-dm
/usr/lib/xen/boot/ioemu-stubdom.gz
/usr/lib/xen/boot/pv-grub-x86_32.gz
/usr/lib/xen/boot/pv-grub-x86_64.gz
[root@slab004 /]#

But i didn't test it :-(

I found a problem when i am check with tap:aio:

disk =[ 'tap:aio:/data/xen-images/vm01/hda.img,xvda,w', ]

Dom0 and DomU will freeze ... I'm on investigation ....

This works stable:
disk =[ 'file:/data/xen-images/vm01/hda.img,xvda,w', ]

@MARK: Have you also installed the new libvirt rpms ?

Cheers,
Sascha
#31 by Mickaël on 9/10/08 - 3:19 AM

Hi,

I'm testing pv-grub with latests RPMs.

I can't get it work with VNC vfb (works fine without it).

Did someone try this ?

This is my VM conf file:

name = "ns1"
maxmem = 512
memory = 256
vcpus = 1
#vfb = [ 'type=vnc,vnclisten=0.0.0.0,vncdisplay=0' ]
disk = [ "drbd:ns1,xvda,w" ]
vif = [ "mac=00:16:3e:34:39:D5,bridge=xenbr0" ]
kernel = "/usr/lib/xen/boot/pv-grub-x86_32.gz"
extra = "(hd0,0)/grub/grub.conf"
#32 by Mickaël on 9/11/08 - 4:41 AM

There is a bug in Xen 3.3.0 that prevent using block-drbd device helper script (and maybe other helper scripts).

You have to apply this patch:

--- /usr/lib64/python2.4/site-packages/xen/xend/server/blkif.py.orig 2008-09-10 09:07:46.000000000 +0200
+++ /usr/lib64/python2.4/site-packages/xen/xend/server/blkif.py 2008-09-10 09:08:13.000000000 +0200
@@ -56,9 +56,9 @@
else:
try:
(typ, params) = string.split(uname, ':', 1)
- if typ not in ('phy', 'file', 'tap'):
+ if typ not in ('phy', 'file', 'tap', 'drbd'):
raise VmError(
- 'Block device must have "phy", "file" or "tap" '
+ 'Block device must have "phy", "file", "tap" or "drbd" '
'specified to type')
except ValueError:
raise VmError(

Cheers
#33 by Sascha on 9/11/08 - 4:48 AM

Hi Mickaël,

i will add this asap and release new rpms again.

Regards,
Sascha
#34 by Sascha on 9/12/08 - 12:48 PM

Hi,

today we checked with pv-grub the first time.

With that, you can use vfb without any problems.
Our test was with SLES-10 SP2 x86_64 as PVM

name = "pvgrub-test"
maxmem = 512
memory = 256
vcpus = 1
vfb = [ 'type=vnc' ]
disk = [ 'file:/var/lib/xen/images/pvgrub-test/hda.img,hda,w' ]
vif = [ "bridge=xenbr0" ]
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0,0)/grub/menu.lst"

Cheers,
Sascha
#35 by Mickaël on 9/15/08 - 4:23 AM

I tried with 64bits VM, but I still can't get vfb work with pv-grub...

I can see grub menu, but vnc server crashes just after (netstat report that it's not listening on the specified port).

I can't any releavant message in any log (xm dmesg, dmesg, /var/log/messages, /var/log/xen/xend.log)

DOes someone have any suggests ?
#36 by Mickaël on 9/15/08 - 10:05 AM

I can't get stubdom working.

My configuraiton files:

# cat xp_pro_sp2_eng|grep -v "^#"|grep -v ^$
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 256
name = "xp_pro_sp2_eng"
vif = [ 'type=ioemu, bridge=xenbr0' ]
device_model = '/usr/lib/xen/bin/stubdom-dm'
boot="dc"
sdl=0
opengl=0
vnc=1
vnclisten="0.0.0.0"
vncpasswd=''
stdvga=0

# cat xp_pro_sp2_eng-dm|grep -v "^#"|grep -v ^$
kernel = "/usr/lib/xen/boot/ioemu-stubdom.gz"
vif = [ '', 'type=ioemu, bridge=xenbr0' ]
disk = [ 'file:/tmp/xp_pro_sp2_eng.img,hda,w', 'file:/tmp/XP_PRO_SP2_ENG.iso,hdc:cdrom,r' ]

* xp_pro_sp2_eng starts paused
* xp_pro_sp2_eng-dm starts
* xp_pro_sp2_eng is unpaused
* xp_pro_sp2_eng-dm stops

then I have (strange state):

# xm list
Name ID Mem VCPUs State Time(s)
Domain-0 0 15847 8 r----- 4598.5
xp_pro_sp2_eng 8 256 1 ------ 0.0

Does anybody get it work successfully ?
Does anybody knows what I'm doing wrong ?
#37 by Boris on 9/27/08 - 9:18 AM

Mickaël,
To get stubdom working on getco's Xen 3.3 port to CentOS 5.2
Create in /etc/xen files:-

[root@ServerCentOS52 xen]# cat WinXPhvm-stubdom
kernel = "/usr/lib/xen/boot/hvmloader"
builder='hvm'
memory = 512
name = "WinXPhvm"
vif = [ 'type=ioemu, bridge=eth0' ]
device_model = '/usr/lib/xen/bin/stubdom-dm'
boot="c"
sdl=0
opengl=0
vnc=0
# vnclisten="0.0.0.0"
# vncpasswd=''
stdvga=0

[root@ServerCentOS52 xen]# cat WinXPhvm-dm
kernel = "/usr/lib/xen/boot/ioemu-stubdom.gz"
vif = [ ' ', 'bridge=eth0']
vfb = [ 'type=sdl' ]
disk = ['file:/etc/xen/isos/winxp.iso,hdc:cdrom,r','file:/etc/xen/images/disk.img,hda,w','file:/tmp/test,hdb,r']

Per Xen 3.3 Stubdom Release Notes run:-
# mkdir -p /exports/usr/share/xen/qemu
# ln -s /usr/share/xen/qemu/keymaps /exports/usr/share/xen/qemu
# mkdir -p /exports/var/lib
# ln -s /var/lib/xen /exports/var/lib
# /usr/sbin/fs-backend &

Switch from console to another terminal session :-

# xm create WinXPhvm-stubdom
#38 by Boris on 9/27/08 - 9:27 AM

In original hvm profile both vnc and sdl are disabled.
Selection is made *-dm Domain's profile.
Just follow carefully release notes.
Install goes hardly alive with long outages , looks like it got stuck
I've set up mem_dom0 to 4GB on 8GB box, but
WinXPhvm-dm allocated only 32 MB, no matter of mem_dom0.
I would guess bad Xen Hypervisor tuning on Dom0 due
to limited experience from my side.
Otherwise, it's worse then usual HVM DomU
#39 by Boris on 9/27/08 - 11:43 AM

That's a final draft.

Getting stubdom working on Xen 3.3 port via http://www.gitco.de/repo/xen3.3.0 to CentOS 5.2

http://bderzhavets.wordpress.com/2008/09/27/gettin...
#40 by Boris on 10/2/08 - 12:49 PM

Ubuntu 8.04.1 & Debian Etch R2 in Stub Domains at Xen 3.3 CentOS 5.2 Dom0 (all 64-bit).
http://bderzhavets.wordpress.com/2008/10/02/ubuntu...
#41 by Sascha on 10/2/08 - 5:16 PM

Hi Boris,

really a great job. Thank you!

Cheers,
Sascha
#42 by Boris on 10/5/08 - 10:32 AM

Attempt to load CentOS 5.2 PV DomU via pv-grub.
GRUB installed in /boot in DomU.

Profile :-
[root@ServerCentOS52 vm]# cat rhel52.pv-grub
name = "rhel-pvgrub"
memory = 2048
vcpus = 1
vfb = [ 'type=vnc' ]
disk = [ 'phy:/dev/sda10,xvda,w!' ]
vif = [ "bridge=eth0" ]
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0,0)/grub/grub.conf"

[root@ServerCentOS52 ~]# xm console rhel-pvgrub
Thread "kbdfront close": pointer: 0x2080130010, stack: 0x15f0000
close fb: backend at /local/domain/0/backend/vfb/28/0
close kbd: backend at /local/domain/0/backend/vkbd/28/0
WARN: No handler for port 5 when unbinding
Booting 'CentOS PVG (2.6.18-92.el5xen)'

root (hd0,0)
Thread "kbdfront close" exited.
Filesystem type is ext2fs, partition type 0x83
kernel /vmlinuz-2.6.18-92.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
................
Hangs

GRUB installed in / in DomU
Profile -
[root@ServerCentOS52 vm]# cat rhel52.pv-grub
name = "rhel-pvgrub"
memory = 2048
vcpus = 1
vfb = [ 'type=vnc' ]
disk = [ 'phy:/dev/sda10,xvda,w!' ]
vif = [ "bridge=eth0" ]
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0,0)/boot/grub/grub.conf"

[root@ServerCentOS52 xen]# xm console rhel-pvgrub
Thread "kbdfront close": pointer: 0x2080130010, stack: 0x15f0000
close fb: backend at /local/domain/0/backend/vfb/24/0
close kbd: backend at /local/domain/0/backend/vkbd/24/0
WARN: No handler for port 5 when unbinding
Booting 'CentOS PVG (2.6.18-92.el5xen)'

root (hd0,0)
Thread "kbdfront close" exited.
Filesystem type is ext2fs, partition type 0x83
kernel /boot/vmlinuz-2.6.18-92.el5xen ro root=/dev/xvda1 rhgb quiet
.............
Hangs

However, for OpenSuse 11 PV DomU pv-grub works fine.
#43 by Boris on 10/5/08 - 11:06 AM

Sorry, GRUB is always in /dev/xvda (MBR for DomU)
Two configs tested :-
1. vmlinuz-xen, initrd-xen in /boot folder.
2. vmlinuz-xen, initrd-xen in / folder (as in OpenSUSE 11 case)
#44 by Boris on 10/5/08 - 4:47 PM

OK. It works. Remove vfb=[ type='vnc'] from profile.

[root@ServerCentOS52 vm]# cat rhel52.pv-grub
name = "rhel-pvgrub"
memory = 2048
vcpus = 1
disk = [ 'phy:/dev/sda10,xvda,w!' ]
vif = [ "bridge=eth0" ]
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra = "(hd0,0)/grub/menu.lst"

Start PV DomU :-

[root@ServerCentOS52 vm]# xm create -c rhel52.pv-grub
------------------------------------------------
GRUB MENU it text mode - Press Enter
------------------------------------------------

Booting 'CentOS (2.6.18-92.el5xen)'

root (hd0,0)
Filesystem type is ext2fs, partition type 0x83
kernel /vmlinuz-2.6.18-92.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-92.el5xen.img

close blk: backend at /local/domain/0/backend/vbd/34/51712
PCI: Fatal: No PCI config space access function found
rtc: IRQ 8 is not free.
i8042.c: No controller found.

CentOS release 5.2 (Final)
Kernel 2.6.18-92.el5xen on an x86_64

rhel52pvm login: root
Password:
Last login: Sun Oct 5 16:26:58 on xvc0
[root@rhel52pvm ~]# vncserver

New 'rhel52pvm:2 (root)' desktop is rhel52pvm:2

Starting applications specified in /root/.vnc/xstartup
Log file is /root/.vnc/rhel52pvm:2.log

[root@rhel52pvm ~]# cat /boot/grub/menu.lst
default=0
timeout=15
splashimage=(hd0,0)/grub/splash.xpm.gz
# hiddenmenu
title CentOS (2.6.18-92.el5xen)
root (hd0,0)
kernel /vmlinuz-2.6.18-92.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet
initrd /initrd-2.6.18-92.el5xen.img
#45 by Boris on 10/6/08 - 8:07 AM

PV-Grub on CentOS 5.2 PV DomU at Xen 3.3 CentOS 5.2 Dom0 (all 64-bit)
Final draft :-
http://bderzhavets.wordpress.com/2008/10/05/pv-gru...
#46 by Sascha on 10/7/08 - 5:35 PM

Hi,

i received a couple of mails, with the question, if the packages are broken ...
Because they get these errors after "yum update xen":

Transaction Check Error:
file /etc/rc.d/init.d/libvirtd from install of libvirt-0.4.4-3 conflicts with file from package libvirt-0.3.3-7.el5
file /etc/sysconfig/libvirtd from install of libvirt-0.4.4-3 conflicts with file from package libvirt-0.3.3-7.el5
file /usr/share/man/man1/virsh.1.gz from install of libvirt-0.4.4-3 conflicts with file from package libvirt-0.3.3-7.el5

Yes, it's normal, because these are only x86_64 packages. You have to remove the i386 version of libvirt.

[root@vh0005 ~]# yum remove libvirt.i386
......

Is this ok [y/N]: y
Downloading Packages:
Erasing : libvirt ######################### [1/1]

Removed: libvirt.i386 0:0.3.3-7.el5
Complete!

[root@vh0005 ~]# yum update xen
..... ####################### [13/13]

Dependency Installed: cyrus-sasl-md5.x86_64 0:2.1.22-4 iscsi-initiator-utils.x86_64 0:6.2.0.868-0.7.el5 qemu.x86_64 0:0.9.0-4
Updated: libvirt.x86_64 0:0.4.4-3 libvirt-python.x86_64 0:0.4.4-3 python-virtinst.noarch 0:0.300.3-7 xen.x86_64 0:3.3.0-3 xen-libs.x86_64 0:3.3.0-3
Complete!

That's all ....

Cheers,
Sascha
#47 by Mickaël Canévet on 10/15/08 - 2:34 AM

Verify that the kernel line in your grub.conf is pointing to /xen.gz-3.3.0.
#48 by Boris on 10/28/08 - 3:42 AM

Install Ubuntu Intrepid Server PV DomU at Xen 3.3 (port via http://getco) CentOS 5.2 Dom0
http://bderzhavets.wordpress.com/2008/10/28/instal...
#49 by Sascha on 11/14/08 - 6:18 PM

Hi all,

today I have built CentOS 5 RPMS (x86_64) for XEN-3.3.1_RC1_pre.

[root@vh0002 ~]# cat /etc/yum.repos.d/XEN-Gitco.repo
[xen]
name=CentOS-$releasever - XEN
baseurl=www.gitco.de/repo/xen3.3.1_rc1_pre" target="_blank">http://www.gitco.de/repo/xen3.3.1_rc1_pre
gpgcheck=0

[root@vh0002 ~]#

[root@vh0002 ~]# yum update
Loading "fastestmirror" plugin
Loading mirror speeds from cached hostfile
* xen: www.gitco.de
* extras: ftp.hosteurope.de
* updates: ftp.plusline.de
* base: ftp.hosteurope.de
* addons: ftp.plusline.de
xen 100% |=========================| 951 B 00:00
primary.xml.gz 100% |=========================| 4.1 kB 00:00
xen : ################################################## 8/8
extras 100% |=========================| 1.1 kB 00:00
updates 100% |=========================| 951 B 00:00
base 100% |=========================| 1.1 kB 00:00
addons 100% |=========================| 951 B 00:00
Setting up Update Process
Resolving Dependencies
--> Running transaction check
---> Package xen-libs.x86_64 0:3.3.1_rc1_pre-0 set to be updated
---> Package xen.x86_64 0:3.3.1_rc1_pre-0 set to be updated
--> Finished Dependency Resolution

Dependencies Resolved

=============================================================================
Package Arch Version Repository Size
=============================================================================
Updating:
xen x86_64 3.3.1_rc1_pre-0 xen 8.8 M
xen-libs x86_64 3.3.1_rc1_pre-0 xen 158 k

Transaction Summary
=============================================================================
Install 0 Package(s)
Update 2 Package(s)
Remove 0 Package(s)

Total download size: 9.0 M
Is this ok [y/N]: y
Downloading Packages:
(1/2): xen-3.3.1_rc1_pre- 100% |=========================| 8.8 MB 00:14
(2/2): xen-libs-3.3.1_rc1 100% |=========================| 158 kB 00:00
Running rpm_check_debug
Running Transaction Test
Finished Transaction Test
Transaction Test Succeeded
Running Transaction
Updating : xen-libs ######################### [1/4]
Updating : xen ######################### [2/4]
Cleanup : xen-libs ######################### [3/4]
Cleanup : xen ######################### [4/4]

Updated: xen.x86_64 0:3.3.1_rc1_pre-0 xen-libs.x86_64 0:3.3.1_rc1_pre-0
Complete!
[root@vh0002 ~]#

[root@vh0002 ~]# xm info
host : vh0002.gitco.tld
release : 2.6.18-92.1.18.el5xen
version : #1 SMP Wed Nov 12 09:48:10 EST 2008
machine : x86_64
nr_cpus : 2
nr_nodes : 1
cores_per_socket : 1
threads_per_core : 2
cpu_mhz : 3200
hw_caps : bfebfbff:20100800:00000000:00000180:0000641d:00000000:00000000:00000000
virt_caps :
total_memory : 3575
free_memory : 3127
node_to_cpu : node0:0-1
node_to_memory : node0:3127
xen_major : 3
xen_minor : 3
xen_extra : .1-rc1-pre
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
cc_compiler : gcc version 4.1.2 20071124 (Red Hat 4.1.2-42)
cc_compile_by : root
cc_compile_domain : gitco.tld
cc_compile_date : Fri Nov 14 23:39:29 CET 2008
xend_config_format : 4
[root@vh0002 ~]#

Cheers,
Sascha
#50 by Boris derzhavets on 11/18/08 - 6:04 AM

Xen 3.3.1rc1-pre port to CentOS 5.2 via http://bits.xensource.com mercurial repos and managing PV DomUs in graphical mode
http://bderzhavets.wordpress.com/2008/11/17/xen-33...
#51 by Larry Patterson on 12/10/08 - 7:24 PM

Great article on Xen 3.2 and Centos 5.2, would you happen to have the same and files for installing Xen 3.3. on CentOS 5.2?

Keep up the good work

Larry
#52 by Kevin on 2/26/09 - 1:24 PM

Hello, thanks for making it easier for the rest of us to use the latest Xen on CentOS 5.2. However, the xen pvSCSI drivers (scsiback and scsifront) appear to be missing. The official Xen 3.3.1 includes these drivers - is there any plan to rebuild your rpm's with pvSCSI support? Thanks...
#53 by tansuo on 3/5/09 - 1:47 AM

hi Ruslan,i have followed what you said to install xen 3.2 on CentOS5.2,but there are some problems. when i reboot my Dell PoweEndger 1950 after " yum install -y xen-3.2.0-0xs.x86_64.rpm xen-libs-3.2.0-0xs.x86_64.rpm xen-devel-3.2.0-0xs.x86_64.rpm",the system halted at :
///////////////////////////////////////////////////////////
Red Hat nash version 5.1.19.6
usb 1-5: device not accepting address 4 ,error -110
usb 1-5: device not accepting address 5, error -110
usb 1-5: device not accepting address 6,error -110
usb 1-5: device not accepting address 7,error -110
/////////////////////////////////////////////////////////

additional infomation :
1. i install centos5.2 on Dell PowerEndger 1950 and download xen-3.3-testing.hg with hg clone
command . I compiled and installed xen followed its README ,but I reboot the system also
halted at :
///////////////////////////////////////////////////////////
Red Hat nash version 5.1.19.6
usb 1-5: device not accepting address 4 ,error -110
usb 1-5: device not accepting address 5, error -110
usb 1-5: device not accepting address 6,error -110
usb 1-5: device not accepting address 7,error -110
/////////////////////////////////////////////////////////

2. I installed fedora8 on the same Dell PowerEndger 1950 and download the xen-3.3-testing.hg
then I compiled and install xen . when i reboot ,the xen-dom0 can work correctly.

the follow infomation is the result of lspci command:
00:00.0 Host bridge: Intel Corporation 5000X Chipset Memory Controller Hub (rev 12)
00:02.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 2 (rev 12)
00:03.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 3 (rev 12)
00:04.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x8 Port 4-5 (rev 12)
00:05.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 5 (rev 12)
00:06.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x8 Port 6-7 (rev 12)
00:07.0 PCI bridge: Intel Corporation 5000 Series Chipset PCI Express x4 Port 7 (rev 12)
00:08.0 System peripheral: Intel Corporation 5000 Series Chipset DMA Engine (rev 12)
00:10.0 Host bridge: Intel Corporation 5000 Series Chipset FSB Registers (rev 12)
00:10.1 Host bridge: Intel Corporation 5000 Series Chipset FSB Registers (rev 12)
00:10.2 Host bridge: Intel Corporation 5000 Series Chipset FSB Registers (rev 12)
00:11.0 Host bridge: Intel Corporation 5000 Series Chipset Reserved Registers (rev 12)
00:13.0 Host bridge: Intel Corporation 5000 Series Chipset Reserved Registers (rev 12)
00:15.0 Host bridge: Intel Corporation 5000 Series Chipset FBD Registers (rev 12)
00:16.0 Host bridge: Intel Corporation 5000 Series Chipset FBD Registers (rev 12)
00:1c.0 PCI bridge: Intel Corporation 631xESB/632xESB/3100 Chipset PCI Express Root Port 1 (rev 09)
00:1d.0 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #1 (rev 09)
00:1d.1 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #2 (rev 09)
00:1d.2 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset UHCI USB Controller #3 (rev 09)
00:1d.7 USB Controller: Intel Corporation 631xESB/632xESB/3100 Chipset EHCI USB2 Controller (rev 09)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev d9)
00:1f.0 ISA bridge: Intel Corporation 631xESB/632xESB/3100 Chipset LPC Interface Controller (rev 09)
01:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS 1078 (rev 04)
02:00.0 PCI bridge: Broadcom EPB PCI-Express to PCI-X Bridge (rev c3)
03:00.0 Ethernet controller: Broadcom Corporation NetXtreme II BCM5708 Gigabit Ethernet (rev 12)
04:00.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Upstream Port (rev 01)
04:00.3 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express to PCI-X Bridge (rev 01)
05:00.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Downstream Port E1 (rev 01)
05:01.0 PCI bridge: Intel Corporation 6311ESB/6321ESB PCI Express Downstream Port E2 (rev 01)
0e:0d.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev 02)

any idea?
#54 by mia on 4/22/09 - 8:57 AM

Hello ! I have trouble:
[root@IIDvirtuals xend]# service xend status
xend is running
[root@IIDvirtuals xend]#

But when i do
[root@IIDvirtuals xend]# xm list
Error: Unable to connect to xend: No such file or directory. Is xend running?
[root@IIDvirtuals xend]# whereis xm
xm: /usr/sbin/xm /usr/share/man/man1/xm.1.gz

What do ?
#55 by DF on 8/24/09 - 10:26 AM

<a href="http://www.louisvuitton2.com/articles/">Lo... Vuitton</a>

<a href="http://www.louisvuitton2.com/articles/index-2.html... Vuitton</a>

<a href="http://www.louisvuitton2.com/articles/index-3.html... Vuitton</a>

<a href="http://www.louisvuitton2.com/LouisVuitton/cheap-Lo... Vuitton Monogram PM Malle Blanche</a>
<a href="http://www.louisvuitton2.com/LouisVuitton/cheap-Lo... Vuitton Monogram PM Keepall</a>
<a href="http://www.louisvuitton2.com/LouisVuitton/cheap-Lo... Vuitton Stacked Trunks</a>
<a href="http://www.louisvuitton2.com/LouisVuitton/cheap-Lo... Vuitton PM Patch Agenda Cover</a>
#56 by alex on 10/1/09 - 9:51 PM

<a href=http://cards.budgetbusters101.com>Make your own business cards</a>
#57 by alex on 10/1/09 - 10:46 PM

<a href=http://cards.boomerbingo.com>Business Cards</a>
#58 by alex on 10/1/09 - 11:07 PM

<a href=http://cards.carpenterbing.com>Business Cards</a>
#59 by alex on 10/2/09 - 12:44 AM

<a href=http://cards.gameratings09.com>Make your own business cards</a>
#60 by alex on 10/2/09 - 12:56 AM

<a href=http://cards.homeimprovement99.com>Make your own business cards</a>
#61 by alex on 10/2/09 - 1:03 AM

<a href=http://cards.uopvpers.com>Business Cards</a>
#62 by alex on 10/2/09 - 1:09 AM

<a href=http://cards.willeatspuppies.com>Business Card Templates</a>
#63 by Dungeon Fighter Gold on 10/8/09 - 10:08 PM

I knew that she could not <a href="http://www.dungeonfighter.us">buy DFO Gold</a> to win the monster. <a href="http://www.dungeonfighter.us">DFO Gold</a> which that I have dreamed I have more, Asda Story; I have played this game for a long time. Although I have spent more <a href="http://www.dungeonfighter.us">Dungeon Fighter Online Gold</a> in this game, I did not have any regret more, by contraries, in order to have more game gold later and continue to play I have thought a way of gaining <a href="http://www.dungeonfighter.us">Dungeon Fighter Gold</a>.
#64 by fe on 11/12/09 - 2:01 AM

<a href="http://www.efes.com">sedde</a>;
[url=http://www.efes.org]esed[/url]
hkj
#65 by Entropiauniverse ped on 12/16/09 - 8:13 PM

In the game <a href="http://www.entropiauniverse.cc/">Entropiauniverse ped</a>and<a href="http://www.entropiauniverse.cc/">Entropia Universe Gold</a> are very important,you can earn<a href="http://www.entropiauniverse.cc/">Entropia Universe Money</a>,then to buy<a href="http://www.entropiauniverse.cc/">cheap Entropiauniverse ped</a>,you also can speng money to <a href="http://www.entropiauniverse.cc/">Buy Entropia Universe Gold</a>.
#66 by cheap zeny on 12/25/09 - 7:53 PM

<a href="http://www.cheers4game.com/product/Ragnarok_Online... zeny</a> is important in this game. If you do not want to be bullied by the high level of players, you should ear more <a href="http://www.cheers4game.com/product/Ragnarok_Online... zeny</a> which can you get a high level in short time. But it is difficulty for a new player to do this. Yet you are lucky because we sell the <a href="http://www.cheers4game.com/product/Ragnarok_Online... zeny</a>. You can buy <a href="http://www.cheers4game.com/product/Ragnarok_Online... zeny</a> from us. I believe that you can play happier with enough <a href="http://www.cheers4game.com/product/Ragnarok_Online... online zeny</a>.
#67 by BGDFH on 12/25/09 - 10:09 PM

In here I found that all the gold were cheap and the [url=http://www.aionth.com]aion gold[/url] was the cheapest, you can come here to buy [url=http://www.aionth.com]aion gold[/url], you can introduce your friends to come to here to buy the [url=http://www.aionth.com]aion gold[/url] we will give you some preferential, and also you can bought the [url=http://www.aionth.com]aion gold[/url], we can sure that in the market our [url=http://www.aionth.com]aion gold[/url] was the cheapest.
#68 by Jill on 12/25/09 - 10:10 PM

If you hope do eve isk better than before, you should finish the mission of fiesta Gold in time . Please remember that flyff penya can help you protect yourself. And habbo credits are very useful to kill the enemy.
#69 by ???? on 1/6/10 - 9:16 PM

???????????<a href="http://www.d2.dion.ne.jp/~uk1231/">???????... href="http://www.citinet.jp/yokohama/">????????&... href="http://www.citinet.jp/tokyoeast/">???????&... href="http://www.citinet.jp/kawasaki/">???????&l... href="http://www.citinet.jp/yokohama/">?????????... href="http://www.citinet.jp/yokohama/">?????????... href="http://www.optimization.co.jp/card/index.html"... href="http://www.optimization.co.jp/card/index.html"...;???????????????????
#70 by dungeon fighter gold on 1/11/10 - 1:17 AM

very good post.i will come to here next time.
#71 by allods gold on 1/16/10 - 4:12 AM

Today, I earn a lot of <a href="http://www.allodsolgold.com/">allods gold</a>, I put some<a href="http://www.allodsolgold.com/">allods online gold</a> to my friends in the game. For my own use the<a href="http://www.allodsolgold.com/">buy allods gold</a> to buy some equipment, and there are a lot of<a href="http://www.allodsolgold.com/">allods ol gold</a> was deposited in the bank.
#72 by gadddhh on 3/10/10 - 10:35 PM

<A href="http://www.kunndmn.com/nara/">????????<... href="http://www.kunndmn.com/yamaguchi/">???????... href="http://www.kunndmn.com/tottori/">????????&... href="http://www.kunndmn.com/shimane/">????????&... href="http://www.kunndmn.com/okayama/">????????&... href="http://www.kunndmn.com/hiroshima/">???????...;
#73 by aaaaa on 3/17/10 - 3:48 AM

<a href="http://www.enterprisecn.com/china-business.html&qu... Business</a>
<a href="http://www.enterprisecn.com/hongkong-business.html... Kong Business</a>
<a href="http://www.enterprisecn.com/international-business... Business</a>
<a href="http://www.enterprisecn.com/international-business... Opportunities</a>
<a href="http://www.enterprisecn.com/china-company-manufact... Manufacturer</a>
<a href="http://www.enterprisecn.com/china-market-news.html... Market</a>
<a href="http://www.theonlinearts.com/picture.html">...;
<a href="http://www.computerlocals.com/computer-peripheral....</a>
<a href="http://www.computerlocals.com/computer-peripheral.... Peripheral</a>
<a href="http://www.computerlocals.com/electronic-part.html...;
<a href="http://befun.520net.to/jobb">The rich man in the woods</a>
#74 by fifa2 gold on 3/18/10 - 3:03 AM

<a href="http://www.fifaonline2gold.com/">fifa2 gold</a> in the game you may or may not notice. I know vitality is very tempting in <a href="http://www.fifaonline2gold.com/">fifa2 online gold</a>. You may be feeling intelligence in <a href="http://www.fifaonline2gold.com/">buy fifa2 gold</a>. You are going to be getting crab pincers which like <a href="http://www.fifaonline2gold.com/">cheap fifa2 gold</a>, depending on which server you are on. <a href="http://www.fifaonline2gold.com/">fifa2 money</a> can help you get a high level in short time.
#75 by cheap aion kina on 3/18/10 - 9:39 PM

Do you know<a href="http://www.kinaaoin.com/">aion kina</a>?if you play the online game,you will know<a href="http://www.kinaaoin.com/">aion gold</a>is the game gold. In the game,if you had more<a href="http://www.kinaaoin.com/">buy aion gold</a>,you will had a tall level. But if you want<a href="http://www.kinaaoin.com/">cheap aion kina</a>,you can come here and spend a little money to bought<a href="http://www.kinaaoin.com/">buy cheap aion gold</a>.Quickly come here.
#76 by dqwdwq on 4/6/10 - 4:11 AM

this blog is kawaii XD
<a href="http://a-frach.com/">;??? ??</a>
I am sorry for doing this lol
I like to drink<a href="http://b-frach.com/">????</a>; these days.
you know<a href="http://c-frach.com/">?</a>;?
its food for sake.
#77 by sfsa on 4/7/10 - 4:32 AM

You need to create a scientist trumpet <a href="http://www.rumblefighter.org/">Rumble Fighter gold</a>, switch to two hundred <a href="http://www.rumblefighter.org/">buy Rumble Fighter gold</a> and seventy thousand to <a href="http://www.rumblefighter.org/">cheap Rumble Fighter gold</a> buy after the birth of our lab-dimensional control <a href="http://www.rumblefighter.org/">Rumble Fighter money</a>.
#78 by buy aion gold on 5/6/10 - 11:32 PM

<a href="http://www.aiongold.fr/">aion money</a> in the game you may or may not notice. I know vitality is very tempting in <a href="http://www.aiongold.fr/">aion gold</a>. You may be feeling intelligence in <a href="http://www.aiongold.fr/">cheap aion gold</a>. You are going to be getting crab pincers which like <a href="http://www.aiongold.fr/">buy aion gold</a>, depending on which server you are on. <a href="http://www.aiongold.fr/">aion online gold</a> can help you get a high level in short time.
#79 by women jordan true flight on 5/10/10 - 6:40 AM

good wishs for you !`
#80 by dfgd on 6/4/10 - 7:01 AM

Do you know <a href="http://www.swgol.com/">swg credits</a>? If you play the online game, you will know <a href="http://www.swgol.com/">Star Wars Galaxies credits</a> is the game gold. In the game, if you had more <a href="http://www.swgol.com/">swg Gold</a>, you will had a tall level. But if you want to <a href="http://www.swgol.com/">buy swg credits</a>. Quickly come here.
#81 by cheap fifa2 gold on 6/16/10 - 11:49 PM

<a href="http://www.fifaonline2gold.com/">fifa2 gold</a> in the game you may or may not notice. I know vitality is very tempting in <a href="http://www.fifaonline2gold.com/">fifa2 online gold</a>. You may be feeling intelligence in <a href="http://www.fifaonline2gold.com/">buy fifa2 gold</a>. You are going to be getting crab pincers which like <a href="http://www.fifaonline2gold.com/">cheap fifa2 gold</a>, depending on which server you are on. <a href="http://www.fifaonline2gold.com/">fifa2 money</a> can help you get a high level in short time.
#82 by asdw on 6/26/10 - 3:10 AM

1062610817PM<a href="http://www.cabalgold.org/index.shtml">Caba... gold</a> | <a href="http://www.cabalgold.org/index.shtml">Caba... alz</a> | <a href="http://www.cabalgold.org/index.shtml">Caba... money</a> |
#83 by xd on 6/30/10 - 5:49 AM

10630034745PM<a href="http://tera4gold.com/index.shtml">tera gold</a> | <a href="http://tera4gold.com/index.shtml">tera online gold</a> | <a href="http://tera4gold.com/index.shtml">buy tera gold</a> | <a href="http://tera4gold.com/index.shtml">cheap tera gold</a> |
#84 by adw on 7/7/10 - 4:53 AM

1077025220PM<a href="http://www.cabalgold.org/index.shtml">Caba... gold</a> | <a href="http://www.cabalgold.org/index.shtml">Caba... alz</a> | <a href="http://www.cabalgold.org/index.shtml">Caba... money</a> | <a href="http://www.cabalgold.org/index.shtml">chea... Cabal Alz</a> |
#85 by xd on 7/8/10 - 1:31 AM

1078113119AM<a href="http://www.cabalgold.org/index.shtml">Caba... gold</a> | <a href="http://www.cabalgold.org/index.shtml">Caba... alz</a> | <a href="http://www.cabalgold.org/index.shtml">Caba... money</a> | <a href="http://www.cabalgold.org/index.shtml">chea... Cabal Alz</a> |
#86 by assdfdfds on 7/8/10 - 3:49 AM

1078015140PM<a href="http://www.aikagold4u.net/">aika gold</a> | <a href="http://www.aikagold4u.net/">aika online gold</a> | <a href="http://www.aikagold4u.net/">cheap aika gold</a> | <a href="http://www.aikagold4u.net/">aika money</a> |
#87 by DVD Ripper on 7/15/10 - 2:04 AM

Your essay makes my life full of sunshine, please.
#88 by menguccishoes on 7/16/10 - 4:15 AM

<a href=\"http://www.menguccishoes.com">men gucci shoes</a>
<a href="http://www.menguccishoes.com">men gucci </a>
<a href="http://www.menguccishoes.com">kids gucci shoes</a>
<a href="http://www.menguccishoes.com">casual gucci shoes</a>
#89 by dsfgd on 7/19/10 - 2:59 AM

10719010650PM<a href="http://www.apbgold4u.com/">All Points Bulletin cash</a> | <a href="http://www.apbgold4u.com/">APB Cash</a> | <a href="http://www.apbgold4u.com/">All Points Bulletin Gold</a> | <a href="http://www.apbgold4u.com/">APB Gold</a> |
#90 by tiffany outlet on 8/3/10 - 9:52 AM

nice post!

i like it!

thanks you!

i like this websit
#91 by icemi on 8/9/10 - 5:41 AM

zxczxc<a href="http://www.rrrbattery.com/">Laptop Battery</a>
#92 by SDFSD on 8/9/10 - 6:15 AM

1089041653PM<a href="http://silkroad4gold.com/">Silkroad Gold</a> | <a href="http://silkroad4gold.com/">SRO Gold</a> | <a href="http://silkroad4gold.com/">Silkroad online gold</a> |
#93 by SDFSD on 8/9/10 - 7:19 AM

1089052054PM<a href="http://www.aikagold4u.net/">aika gold</a> | <a href="http://www.aikagold4u.net/">aika online gold</a> | <a href="http://www.aikagold4u.net/">cheap aika gold</a> | <a href="http://www.aikagold4u.net/">aika money</a> |
#94 by manolo blahnik online store on 8/14/10 - 8:22 AM

That is a idea, I am agree with the opinions of this articles about
#95 by waner on 8/25/10 - 5:05 AM

<a href="http://www.warner-tech.hk">dual sim Watch phones</a>
#96 by suprashoesstore on 8/25/10 - 11:02 AM

<h1>[url=http://www.airmax-shoes.us/womens-air-max-c-46.htm...]<strong>womens max air</strong>[/url]</h1>
<h1>[url=http://www.suprashoesonline.us/supra-skytop-ii-c-9...]<strong>Supra Skytop II</strong>[/url]</h1>
#97 by fhgfhbgf on 8/28/10 - 1:42 AM

10828115345AM<a href="http://12sky2gold4u.com/">twelvesky2 gold</a> | <a href="http://12sky2gold4u.com/">12sky2 gold</a> | <a href="http://12sky2gold4u.com/">12Sky2 Silver</a> | <a href="http://12sky2gold4u.com/">buy 12Sky2 Silver Coins</a> |
#98 by SDSD on 9/4/10 - 5:06 AM

1094030739PMhttp://www.aikagold.cc/index/ aika gold | http://www.aikagold.cc/index/ aika online gold | http://www.aikagold.cc/index/ cheap aika gold | http://www.aikagold.cc/index/ aika money |
#99 by vibram 5 fingers on 9/7/10 - 4:56 AM

<h1><a href="http://www.vibramfivefingerstoday.us"><... fingers vibram</strong></a>, five fingers vibram</h1><br><br />
<h1><a href="http://www.vibramfivefingerstoday.us"><... five finger</strong></a>, vibram five finger</h1><br><br />
<h1><a href="http://www.vibramfivefingerstoday.us"><... 5 fingers</strong></a>, vibram 5 fingers</h1><br><br />
#100 by ukreplicas on 9/9/10 - 11:53 PM

Cheap replicas wholesale !
#101 by hollister clothing on 9/10/10 - 2:27 AM

Wish there’s a way out for all of us, is there? I hope so.
#102 by sdas on 9/13/10 - 6:19 AM

10913041859PM[url=http://flyffgold.org/]flyff penya[/url] | [url=http://flyffgold.org/]flyff gold[/url] |
#103 by asdf on 9/15/10 - 4:41 AM

10915024111PM<a href="http://www.aika4gold.com/">aika gold</a> | <a href="http://www.aika4gold.com/">aika online gold</a> | <a href="http://www.aika4gold.com/">cheap aika gold</a> | <a href="http://www.aika4gold.com/">aika money</a> |
#104 by P90x on 9/16/10 - 3:56 AM

This is a great post; it was very informative. I look forward in reading more of your work. Also, I made sure to bookmark your website so I can come back later. I enjoyed every moment of reading it. internet marketing
#105 by moncler Boutique on 9/16/10 - 4:08 AM

I provide information that is truly unique and the service they offer too have lived, I love this service so I request please visit the others.
#106 by wsdq on 9/19/10 - 5:52 AM

10919035210PM[url=http://www.at4gold.com/index.shtml]Atlantica Online Gold[/url] | [url=http://www.at4gold.com/index.shtml]Atlantica Gold[/url] |
#107 by ugg women on 9/20/10 - 4:43 AM

When talking about women''s boots, accurately ugg bailey button triplet , you can use a brace of Hunter Wellies for muddy, sunny, and waterproof boots back Aussies absolutely adulation adequate the calefaction of the sun at an advantageous brace of womens boots,
#108 by bailey ugg boots uk on 9/20/10 - 6:30 AM

When I was asked to give a write-up, on the bailey ugg boots uk topic “Life is Beautiful” I smiled. I believed it was a simple topic with a very simple uggs bailey button boots proposition. With a firm belief in myself, I tried to pen a few cheap ugg boots argyle words. I found myself helpless as I had fiddled away precious ugg argyle knit time. Apparently an innocuous proposition made me ponder, which began in a listless uggs classic cardy way and later took a definite direction.As my thought process gained some ugg classic cardy tall ground, I could not fathom the depth of this topic nor scale the height of it. At one sundance ugg boots time I thought it could be dealt by filling the write up with ugg sundance boots anecdotes of my life and thus proving Life is Beautiful. The very next instance made me shudder, as a serious topic should be dealt tall uggs on sale philosophically. A chain developed with one approach giving way to the other.This approach classic tall uggs closely follows the philosophical ugg boots sale uk approach with a beauty of its own. To appreciate the beauty of life one can relish the uggs boots works of artists and writers of renaissance ugg boots online store uk period. Be it Da Vinci with the ethereal Mona Lisa, Rembrandt or Monet with Water Lillies brought out the essence of ugg boots from usa life. http://www.uggsbailey.co.uk/sitemap.html xy
#109 by bailey ugg boots uk on 9/20/10 - 6:31 AM

When I was asked to give a write-up, on the bailey ugg boots uk topic “Life is Beautiful” I smiled. I believed it was a simple topic with a very simple uggs bailey button boots proposition. With a firm belief in myself, I tried to pen a few cheap ugg boots argyle words. I found myself helpless as I had fiddled away precious ugg argyle knit time. Apparently an innocuous proposition made me ponder, which began in a listless uggs classic cardy way and later took a definite direction.As my thought process gained some ugg classic cardy tall ground, I could not fathom the depth of this topic nor scale the height of it. At one sundance ugg boots time I thought it could be dealt by filling the write up with ugg sundance boots anecdotes of my life and thus proving Life is Beautiful. The very next instance made me shudder, as a serious topic should be dealt tall uggs on sale philosophically. A chain developed with one approach giving way to the other.This approach classic tall uggs closely follows the philosophical ugg boots sale uk approach with a beauty of its own. To appreciate the beauty of life one can relish the uggs boots works of artists and writers of renaissance ugg boots online store uk period. Be it Da Vinci with the ethereal Mona Lisa, Rembrandt or Monet with Water Lillies brought out the essence of ugg boots from usa life. http://www.uggsbailey.co.uk/sitemap.html xy
#110 by p90x workout on 9/21/10 - 2:56 AM

This is a great post; it was very informative. I look forward in reading more of your work. Also, I made sure to bookmark your website so I can come back later. I enjoyed every moment of reading it. internet marketing
#111 by RETRRT on 9/28/10 - 8:15 AM

10928061807PM<a href="http://www.maplestory4mesos.org/">Maplestory</a> | <a href="http://www.maplestory4mesos.org/">Maple Story Mesos</a> | <a href="http://www.maplestory4mesos.org/">MapleStory" target="_blank">http://www.maplestory4mesos.org/">MapleSto... Mesos</a> | <a href="http://www.maplestory4mesos.org/">Maple Story Meso</a> |
#112 by ugg on 10/4/10 - 4:58 AM

When you are ready to buy a pair of <a href="http://www.edhardyhub.com"; ref="dofollow" ><strong>Ed Hardy </strong></a> sunglasses expect to pay $100+ for a pair. If you can afford the real deal we are sure you will be happy with them. If you can not afford <a href="http://www.uggbootsvogue.com/"; rel="dofollow"><strong>UGG boots</strong></a> them then what can you do? we'll we would suggest looking at replica wholesale <a href="http://www.edhardyhub.com/ed-hardy-sunglasses-c-24...; rel="dofollow" ><strong>Ed Hardy Sunglasses</strong></a>. These have cool looking designs on the arms of the sun glasses and look pretty cool for a fraction of the cost of the real ones.<BR>
</DIV>
<p> 
#113 by Online Bargain Shopping on 10/5/10 - 12:44 PM

Give me a low five, cause I can help myself. Give me a low five downsize no one else.
#114 by dvsdv on 10/22/10 - 10:15 PM

101023081439AM<a href="http://romgoldsell.com/">Runes of Magic gold</a> | <a href="http://romgoldsell.com/">rom gold</a> | <a href="http://romgoldsell.com/">cheap rom gold</a> |
#115 by DSDS on 10/23/10 - 4:30 AM

101023023023PM[url=http://www.4story4gold.com/index.shtml]4 Story Gold[/url] | [url=http://www.4story4gold.com/index.shtml]4 Story Luna[/url] |
#116 by ascasc on 10/29/10 - 11:36 PM

101030093612AM<a href="http://12sky2gold4u.com/">twelvesky2 gold</a> | <a href="http://12sky2gold4u.com/">12sky2 gold</a> | <a href="http://12sky2gold4u.com/">12Sky2 Silver</a> | <a href="http://12sky2gold4u.com/">buy 12Sky2 Silver Coins</a> |
#117 by DSAD on 11/8/10 - 3:26 AM

10118022621PM[url=http://www.buyarchlordgold.net/index.shtml]Archlord Gold[/url] | [url=http://www.buyarchlordgold.net/index.shtml]Buy Archlord Gold[/url] |
#118 by GHJNGJNHGGH on 11/15/10 - 4:29 AM

101115033017PMhttp://www.4story4gold.com/index.shtml 4 Story Gold | http://www.4story4gold.com/index.shtml 4 Story Luna | http://www.4story4gold.com/index.shtml cheap 4story luna | http://www.4story4gold.com/index.shtml buy 4story gold |
#119 by jhvhjvj on 12/13/10 - 2:49 AM

101213014857PM[url=http://www.buy4storyluna.com/]4 Story Gold[/url] | [url=http://www.buy4storyluna.com/]4 Story Luna[/url] | [url=http://www.buy4storyluna.com/]cheap 4story luna[/url] | [url=http://www.buy4storyluna.com/]buy 4story gold[/url] |
#120 by sdvd on 12/14/10 - 4:13 AM

101214031314PM[url=http://www.buy4storyluna.com/]4 Story Gold[/url] | [url=http://www.buy4storyluna.com/]4 Story Luna[/url] | [url=http://www.buy4storyluna.com/]cheap 4story luna[/url] | [url=http://www.buy4storyluna.com/]buy 4story gold[/url] |
#121 by Emal on 6/30/11 - 5:45 AM

store which sell http://www.ghdcheap-australia.com welcome!.
#122 by BBCnowBSB on 8/22/11 - 8:59 AM

Is simply not good enough!! BBC have a duty to respond to people asking questions, nobody has answered a question thye have just repeated the same statement. This is not going to go away by just ignoring it!
#123 by Uggs Cheap For Sale on 10/3/11 - 8:10 AM

BBC have a duty to respond to people asking questions, nobody has answered a question thye have just repeated the same statement.
#124 by YusnD on 10/13/11 - 7:20 AM

The BBC is arrogant and has wasted money, it is unable to afford F1 because having overspent on grandiose schemes there's no money in the bank. Can we reverse this fact of life? No. If the Contract has been signed we can't reverse that either - no doubt there are massive penalty clauses.
#125 by eric320 on 10/14/11 - 6:25 AM

good articles,thanks for sharing .