QEMU vs. VirtualBox

Why is it that the U.S. Government always releases a slew of RFPs just before Thanksgiving?  I’ve been swamped working on proposals since the third week of November, but we got the last one submitted just before Christmas so it’s back to normal (or what passes for normal around here) for a while.

I thought I’d take this relatively quiet period to do a quickie comparison between a couple of virtualization tools: QEMU and Oracle's VirtualBox.  For the comparison I chose to install virtual guest instances of Ubuntu 10.10 desktop from a downloaded copy of the iso.  The host system is an AMD 64-bit machine that is also running Ubuntu 10.10 desktop. Here’s the kernel version of the host at the time of this writing:  2.6.35-24-generic #42-Ubuntu SMP Thu Dec 2 02:41:37 UTC 2010 x86_64 GNU/Linux. I used VirtualBox 4.0 for this comparison.

Both products provide a graphical interface for building a new virtual guest instance.  Here are what a few of the QEMU build sequence steps look like:

 


 

 

 

 This review almost did not get written, because it took friggen forever for the QEMU install to complete.  It was only out of a somewhat morbid fascination that I let it go to completion.  I started the install at about 8:15am in the morning, and by 1:00pm it was finally finished.  By comparison, the VirtualBox install took just 28 minutes start to finish, and that included download time for updates,  since I had selected the option to do that at install time.  I did not select the update option for the QEMU install because I (fortunately) forgot to select NAT networking for it prior to starting the installation.

After the QEMU install was done, the Ubuntu guest remained equivalently slow.  Boot time was about 6 minutes as compared to 23 seconds for the VirtualBox Ubuntu guest.  Once I got the logon prompt from the QEMU guest it took a very long time to achieve the final QEMU result:


 

That’s right, a black screen. Which lasted so long I thought that was the final product.  About 15 minutes later, though, I got a login prompt, which promised to take forever to execute, so I finally put it out of its misery.


Oracle does not need to be too worried about competition from the QEMU camp just yet...

------------------------------------------------  UPDATE: 1/15/2011 -- The performance problem has been fixed by completely removing the kvm  and qemu-kvm packages and then reinstalling.  Thanks to all the comments which helped me zero in on the problem.  Also, at the suggestion of one reader, I reset the BIOS option for enabling virtualization and then power cycled the host.  Not sure if that was necessary, but I did it anyhow.
Performance of the VMs are now as others have reported, with install times taking minutes rather than hours.
--Doug 



 

 

 

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Support Virtio by virt-manager

dbaxps's picture

> I thought you could only do that with virsh -- virt-manager does not support virtio in its interface
That's wrong. Step 2 of 5
OS Type : Linux
Variant : Generic 2.6.25 or later kernel with virtio
Another option via virt-install. View :-
http://www.linuxtoday.com/it_management/2011010100635OSSV

By the numbers...

JohnG's picture

I am using Virtual Machine Manager 0.8.4 from RH.
In Step 4 of 5 of the new machine wizard, I
[x] Enable Storage for Virtual Machine
(o) Create a disk image on the computer's hard drive
[ 8.0 ] GB
(o) Allocate entire disk now.

After the VM creation, I look in the details of the VM at VirtIO Disk 1 and see it's the 8GB disk file I created and allocated earlier. I'm assuming that means I'm using virtio for that drive.

I'm storing the virtual images on an older 5400 rpm 500GB SATA drive in the same machine as the VM host (i.e. not over NFS or a SAN connection).

As far as host OS, we use CentOS 5.5 as a host OS in the production environment, but I use Kubuntu at home. I'm not really one for vendor religious wars, I just use one until it disappoints me and then move to the next Aside from a little churn, the Ubuntu stuff has disappointed me least, lately.
From my limited experience, virtualization : RAM as real-estate : location.

Good luck.
John

Thanks for the clarification

Doug.Roberts's picture

I used exactly the same process when I created my ubuntu guest with virt-manager.

--Doug

While qemu is not as fast as

Anonymous's picture

While qemu is not as fast as virtualbox by using virtualization it is pretty much usable.

When I read "Testing with kvm will be the subject for another article." then I think you should try virtualbox without virtualization too.
That virtualbox uses it "by default" is not a very good excuse because qemu needs you to set many parameters "by default". It's just another way of using programs. Per default it uses 256 mb ram. I'm pretty sure you changed this on virtualbox too.

This is an impression of how it should work. For some reason my mouse didn't work. This is the first time it didn't work with qemu, no idea why.
http://dl.dropbox.com/u/13721560/qemu-ubuntu.mkv

The main difference between qemu and virtualbox are the graphics drivers. For me qemu is always a bit slow and laggy with drawing the mouse pointer or windows, no matter what virtual video card I choose (I have tried all of them). On virtualbox it really flies.
And, windows xp is booted in 3-5 seconds in virtualbox (4). Virtualization finally is usable for everyday use (if you have VT).

Hmm

David Lane's picture

Well, I have to say I have had a completely different experience, but I also have different hardware.

I am running KVM with the hardware virtualization enabled in the CPU, which makes things a little bit more like apples and oranges. I am also running RedHat rather than Ubuntu as both the Host and Guest OSs. Finally I am running bridged networking rather than NAT because of speed issues on the interface.

From what I have experienced, all performance metrics are in line EXCEPT the SOFTIRQ value which seems to be running high on the Guest. While this is expected, I also have some tuning to perform and I expect the value to come down to more than one standard deviation from mean (it is currently much, much higher).

The paravirtualization, offered by VirtualBox, and the Virtual Network model, on the other hand resulted in a significant speed issue, both in the application on the guest as well as traffic through the interface at the host and the guest. Because the tested application was more than just your basic file and print, and throughput was critical, and measurable, it made VirtualBox a non-starter.

It really comes down to ensuring that you are testing the right combinations of hardware and software. I have found that it really depends on your needs and set up. Ubuntu on Fedora with a hardware KVM solution, as well as Windows XP was beastly slow and and unresponsive in a number of situations, but Windows 7, RHEL and Fedora on the same hardware platform seem to perform well. Conversely, Ubuntu and XP seemed to perform well on Virtual Box on the Fedora system while Win7 was unusable. I never bothered to dig into the why. Maybe in my spare time.

For the curious: RHEL6 x86 running on Dell R610 6 core, 6GB RAM with hardware virtualization enabled is one of my configuration while my laptop is a Dell M4400 Dual core with 8 GB RAM running Fedora 13 x64. I will be upgrading the RHEL to 64-bit moving forward.

David Lane, KG4GIY is a member of Linux Journal's Editorial Advisory Panel and the Control Op for Linux Journal's Virtual Ham Shack

I was a bit surprised myself

Doug.Roberts's picture

David,

I was fully expecting the KVM/qemu system to perform equivalently to VirtualBox or VMWare on my system, based on what I've read. FYI I also tried to get Windows XP installed as a qemu VM, but it too was unmanageably slow.

My suspicions are that something in the kvm.ko or kvm_amd.ko kernel modules are broken in my current kernel version. I'll keep an eye on the kernel developer sites to see if I can verify this.

--Doug

XP and Virtio Drivers

Kevin Bush's picture

I have found virtio drivers to be essential with XP guests on qemu/kvm. With those drivers, performance was equivalent to virtualbox on my systems. My main issue was crawling disk io without virtio drivers.

Thanks,

Kevin

Kevin Bush is a Linux systems admin, dad and book-lover who spends far too
much time tinkering with gadgetry.

Interesting article

whereisian's picture

Even more interesting comments.

It would seem that QEMU is not using hardware virtualization for whatever reason. It doesn't make sense that there would be such a difference in performance when everything else I read says that performance is comparable.

I look forward to seeing followups.

kvm/qemu confusion & networking

Roland's picture

Why are kvm and qemu involved with each other? This causes massive confusion. Qemu needs a decent tutorial. Virtualbox suffers from being limited to a GUI. Qemu is much more versatile. But networking doesn't work out of the box with qemu, and I haven't been able to get it to work past DNS. That's right 'ping google.com" resolves to an IPaddress, end of story. Qemu is fast and flexible once you get kvm and kvm-amd involved, but it badly needs polish. Virtualbox, on the other hand, works from the start but is a PITA to use.

VirtualBox limited to a GUI?

crharper's picture

VirtualBox does not need the GUI. Any Guest can be fully configured via the command line.

I'm convinced

8forty's picture

Since you obviously have some knowledge of hardware, software, and Ubuntu, I'm convinced that Virtualbox is a better choice than QEMU+KVM for the user who wants an easy-to-use and good performance desktop VM with minimum effort. Your addition of some mention of the extra research you did to try to get QEMU+KVM working supports my conclusion.

@Anonymous: obviously QEMU+KVM doesn't work "out-of-the-box" nearly as well as Virtualbox, unless you're assuming Mr. Roberts is unusually inept (which seems unlikely to me based only on this article). He obviously didn't have to "Ask for help ... [to] figure it out" with Virtualbox.

Not my experience

Josiah's picture

Virtualbox isn't OSS all the way to the end. Extensions that are somewhat important imho aren't under a traditional OSS licences and, in my opinion, Oracle hasn't proven themselves as OSS friendly.

Besides, I think you're comparing apples to oranges. We have one solution that caters to desktops more and another that caters more to the full virtual server environment. Both can work in the direction of the other, but... Anyway. I think that's key to your experience.

I'm running a qemu+kvm virtual server with the appropriate hardware and find it to be plenty sufficient for my needs running production servers. Start-up times are definitely measure in seconds not minutes as you experienced. Ubuntu is running on my host and some guest machines. I'm pulling the storage for the VMs from iSCSI and managing them remotely with virt-manage, which is the buggy part of it. I'm thinking about giving up and just going with virsh instead. Otherwise, my experience is quite different.

I'd suggest that the hardware may be insufficient or the configuration not as good. Perhaps a comparison on Ubuntu server with the virtual machine host option in install would make some significant difference for you.

try virtio

lims's picture

You probably need to choose virtio block devices. I had the same issue. Slow VMs that became peppy after configuring them to be virtio. There are drivers for Linux & Win. FreeBSD drivers are in the works.

You lost me

Doug.Roberts's picture

I'm afraid that I did not understand your comment. Where in the VM setup process is this choice made?

--Doug

I suspect you made 2 errors.

Chris Cowley's picture

I suspect you made 2 errors. 1) You used basic Qemu on its own, rather than using the kqemu module, or even better KVM (which need Virtualisation extensions in the CPU). 2) You did not use virtio.

Quite frankly, if you knowledge of Linux Virtualisation is this limited, why are you commenting on it? I use KVM at work and it is as fast as Xen, and wipes the floor with VMware.

If you do not understand what these concepts are then please re-consider what you write about (Windoze maybe?) - this is laughable.

win32 virtio

lims's picture

I don't believe it's on the

lims's picture

I don't believe it's on the interface yet. Virsh (the shell equivalent) however, should be able to support it. For example, on my shell:

# virsh version
Compiled against library: libvir 0.8.1
Using library: libvir 0.8.1
Using API: QEMU 0.8.1
Running hypervisor: QEMU 0.12.5

* You may need to create a block device to install your guest OS. I have LVM, so this is an example
lvcreate -L 15G -n /dev/lv01/guest_1.fs

* Then bootstrap your install
# virt-install --name=virtio_template \
--ram 1024 \
--os-type linux --os-variant debiansqueeze \
--disk path=/dev/lv01/guest_1.fs,bus=virtio \
--network network=default,model=virtio \
--vnc \
--noautoconsole \
--cdrom /home/lmwangi/debian-testing-amd64-netinst.iso \
--virt-type kvm \
--force yes

The GUI client is probably doing something akin to:

virt-install --name=virtio_template \
--ram 1024 \
--os-type linux --os-variant debianlenny \
--file /dev/lv01/guest_1.fs \
--network network=default \
--vnc \
--noautoconsole \
--hvm \
--cdrom /home/lmwangi/debian-505-amd64-netinst.iso
--force yes

Which was incredibly slow.

PS: You can also convert your existing VMs by trying virsh edit
Mine looks like:
....

/usr/bin/kvm

...

......

I hope this helps

Yes

Doug.Roberts's picture

This helps a lot, thanks.

--Doug

I thought VirtualBox and KVM/QEMU colide...

Rock's picture

(I'm sorry I don't have the source.) I read somewhere that these two will have issues running on the same box.

I recently used the KVM virt-manager to install two VM's, one was Ubuntu the other Fedora 14. I had the Fedora VM up and running in less than one hour; the Ubuntu install took a very long time, at least 6 hours - I went to bed while it chugged away. Like you, I also find that the boot up of the VM's takes a long time and the response is not acceptable for my needs. (I wonder if I have VirtualBox installed and it is stepping on my KVM?...)

I wish to find a usable headless solution - while wishing, I also wish to have more time finding a solution; I digress...

I greatly look forward to reading about your VM adventures and the constructive feed back of your readers.

Rock

Followup/Correction...

Rock's picture

It appears that I do not have VirtualBox installed on my host, so the slow VMs (KVM/QEMU) can not be attributed to any conflict between them.

Correction: I did not use virt-manager to create the VMs, I used the command line virt-install program. (The same command from Linux Journal Issue 201 page 75 - "Managing KVM Devpoyments with Virt-Manager".)

After reading this post and the user communities comments, I will be removing all traces of KVM/QEMU and installing VirtualBox and see if I get better results.

Follow up to my followup....

Rock's picture

Short version: I have my KVM/QEMU VM working much faster now!!! It is actually very usable. (However, it did require much more digging/googling than I would have liked.)

Long version: (My host is modified Fedora-12 if that matters.)
I did not give up on KVM/QEMU like I said I was going to do... Since I'm using a quad-core Intel cpu, I did the 'grep vmx /proc/cpuinfo' and since four lines were returned I made the bad assumption that my host was using the VT hardware. I found out ( thanks to https://wiki.ubuntu.com/kvm ) that my bios had VT hardware disabled - thank you Intel... After fixing that, my VMs were still very, very slow... I deleted my Ubuntu 10.04.1 VM and recreated it with:

virt-install --connect=qemu:///system \
--name=ubuntu-10.04.1 \
--ram=1024 \
--vcpus=2 \
-f /storage/VMs/KVM/ubuntu-10.04.1.qcow2 \
-s 24 \
--cdrom=/storage/VMs/KVM/ISOs/ubuntu-10.04.1-desktop-i386.iso \
--vnc \
--virt-type=kvm \
--noautoconsole \
--accelerate \
--os-type=linux \
--os-variant=generic26 \
--network=bridge:br0

I would really like to use virtio, but I am not using VLM, so I was unable to use a previous posters advice. I'm tired of reading and researching on how to get KVM/QEMU tweeked to the point of usable...

The new install only took about 20 minutes compared to the hours it took before!!!

As you can tell, I am not a VM expert and I have now desire to be one. With that in mind, it would be very helpful if the software would have given me a hint to check my bios since dmesg knew my bios had it disabled...

Stop reading here! I'm very surprised at some of the negative attitude here, it is not what I am accustomed too in the linux world where people help others. I hope this is the exception and not the norm. I hope fanboys, fanatics and trolls do not kill this thread.

Follow up to my followup....

Rock's picture

Short version: I have my KVM/QEMU VM working much faster now!!! It is actually very usable. (However, it did require much more digging/googling than I would have liked.)

Long version: (My host is modified Fedora-12 if that matters.)
I did not give up on KVM/QEMU like I said I was going to do... Since I'm using a quad-core Intel cpu, I did the 'grep vmx /proc/cpuinfo' and since four lines were returned I made the bad assumption that my host was using the VT hardware. I found out ( thanks to https://wiki.ubuntu.com/kvm ) that my bios had VT hardware disabled - thank you Intel... After fixing that, my VMs were still very, very slow... I deleted my Ubuntu 10.04.1 VM and recreated it with:

virt-install --connect=qemu:///system \
--name=ubuntu-10.04.1 \
--ram=1024 \
--vcpus=2 \
-f /storage/VMs/KVM/ubuntu-10.04.1.qcow2 \
-s 24 \
--cdrom=/storage/VMs/KVM/ISOs/ubuntu-10.04.1-desktop-i386.iso \
--vnc \
--virt-type=kvm \
--noautoconsole \
--accelerate \
--os-type=linux \
--os-variant=generic26 \
--network=bridge:br0

I would really like to use virtio, but I am not using VLM, so I was unable to use a previous posters advice. I'm tired of reading and researching on how to get KVM/QEMU tweeked to the point of usable...

The new install only took about 20 minutes compared to the hours it took before!!!

As you can tell, I am not a VM expert and I have now desire to be one. With that in mind, it would be very helpful if the software would have given me a hint to check my bios since dmesg knew my bios had it disabled...

Stop reading here! I'm very surprised at some of the negative attitude here, it is not what I am accustomed too in the linux world where people help others. I hope this is the exception and not the norm. I hope fanboys, fanatics and trolls do not kill this thread.

One thing you might have

Dim's picture

One thing you might have missed: when you enable VT in bios, it is not enough to save the changes and reboot, you actually have to powercycle the machine. This is a pit I've fallen into myself before figuring it out.

Hope this helps.

Thanks

Doug.Roberts's picture

I did not know that. However, I did verify that the bios virtualization setting was enabled, so power cycling the machine isn't required. BTW, as a test I disabled the bios setting and then tried to load the kvm kernel modules. I discovered that kvm-amd cannot load with the bios virt setting disabled.

--Doug

VirtualBox Virtues

crharper's picture

Don't forget to mention that VirtualBox supports 'headless' operation allowing connection via any Remote Desktop Client. A big plus if the OS itself doesn't support RDP or VNC! The Guest Additions allow for resizing the host's desktop. Boot times are very respectable too.

Has anyone done anything with Promox VE which supports VNC?

Ok, but what about

risca's picture

I ran out of time

Doug.Roberts's picture

Testing with kvm will be the subject for another article.

--Doug

seriously?

Anonymous's picture

virtualbox automatically installs a kernel module, so if you haven't installed kvm for qemu, then you're comparing apples to oranges. the conclusion that qemu is slow is painfully should have been obvious; this article didn't need to be written. if qemu can't get access to hardware virt, then its going to revert to software virt, and that is obviously going to be much slower.

Kvm for qemu was installed

Doug.Roberts's picture

The following packages were installed:

kvm
qemu-kvm
libvirt0
phthon-libvirt
libvirt-bin
phthon-vm-builder

The kvm kernel module is also present:

roberts@igor:~$ locate kvm.ko
/lib/modules/2.6.35-24-generic/kernel/arch/x86/kvm/kvm.ko

--Doug

was it actually loaded?

Anonymous's picture

$ lsmod | grep kvm

and if not load it with

$ sudo modprobe kvm

Yes

Doug.Roberts's picture

roberts@igor:~$ lsmod | grep kvm
kvm_amd 40563 0
kvm 298966 1 kvm_amd

...but nothing is using it.

davidwc's picture

...but nothing is using it. Note the 0.

This article is ridiculous. Your conclusion is based on a poor knowledge of the subject and anyone who doesn't read past the article to get to the comments will be deeply misinformed. As I and many others can attest to, qemu+kvm is rock solid.

Nothing is using them,

Doug.Roberts's picture

because the qemu guest is not running.

And, your insulting behavior does you a discredit. What's your real name?

QEMU requires CPUs that support hardware virtualization

Anonymous's picture

Hardware virtualization

Doug.Roberts's picture

My processor (AMD Athlon II X2 250 Regor 3.0GHz Socket AM3 65W Dual-Core) reports that it supports svm virtualization:

cat /proc/cpuinfo | grep svm
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt npt lbrv svm_lock nrip_save

VirtualBox seems capable of taking advantage of this.

--Doug

apples to apples?

Dim's picture

Pure QEMU is _only_ an emulator. It cannot take advantage of VT/AMD-V without KVM, and is actually not supposed to.
So if you want to compare speeds, test with KVM, or test on a system with no AMD-V at all.

If you do want to compare QEMU to anything, look for another multiple CPU architecture emulator, and compare there. You will not be able to bring up an ARM OS on VBox, or a MIPS based OS for example. But you can easily do that on QEMU, because that is what it's for.

You don't have "hardware

Anonymous's picture

You don't have "hardware virtualization" per your title. You really need to make sure that you understand the requirements and have qemu+kvm configured correctly before publishing this type of artice. It's not difficult to enable KVM and a quick google of "kvm ubuntu" would provide you with numerous examples. By not doing your homework, you've done a disservice to KVM and QEMU.

A "quick google" ?

Doug.Roberts's picture

I spent several days doing "long googles" for documentation on how to get qemu and kvm running on an Ubuntu host. What you see is the result of that exercise.

Perhaps there was one magic "quick google" link that I missed, but I think not.

However, since my hardware supports virtualization, and since I did substantial research on the topic before writing the topic up, I'd like to suggest another explanation for the poor performance of the qemu guest OS on my system: qemu/kvm is not quite mature enough yet for everyday use.

too difficult to type "modprobe kvm_amd" ?

Anonymous's picture

Not sure how you would have missed that QEMU hardware virtualization requires a kvm module during your "substantial research." Installing a few packages with apt-get install and typing a modprobe command (or just rebooting) are not difficult. Really, KVM works "out of the box" on Ubuntu, Fedora, Redhat, and probably others that I haven't tried.

You really need to rewrite this article. Ask for help if you can't figure it out.

You don't read very carefully

Doug.Roberts's picture

Earlier comments addressed the kvm module issue. So save you the effort of actually reading the previous comments, I'll recap here:

roberts@igor:~$ lsmod | grep kvm
kvm_amd 40563 0
kvm 298966 1 kvm_amd

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix