Sun (Now Oracle) VirtualBox: An Observation

Ok, I recognize and greatly appreciate the tremendous contributions of open source software.  I really do.  I use it every day, and have done so for many years.  I’m a Linux person from the word go.  But, I work for a Windows Shop.  All Windows, All The Time.  So, to fill out my time sheet in order to get paid, or to follow all the Track Changes in a Microsoft 2007 proposal document, I have to run Windows, because the company I work for uses 100% Windows software to run their business.

I accept that.  And to ensure that I can fill out my time sheet, and track all those changes in MS Word documents, I run Windows in a virtual machine.  (As an aside, I also use Open Office, have done so for years, but OO is not 100% compatible with MS Word.  It’s also not always too swift with track changes, and I cannot count the hours I have wasted trying to make an OO Presentation look like anything other than the dog’s dinner when viewed in MS Power Point.)

Back to the issue: for my virtual machine, I used to use VMWare, but I switched over to Sun (now Oracle) VirtualBox several years ago.  Generally, I’ve been quite happy with VirtualBox.  I run the latest patched version of Windows XP in it; it’s got 2-D hardware acceleration, sound, virtual networking etc., so you basically can’t tell that you’re on a virtual machine when you’re using it.

However...

Today I ran up against one of those apparently persistent, long-lived bugs in VirtualBox that threatens to derail my productivity for at least a few more hours.  Here’s the background:  I created the VirtualBox VM that I am currently using about 3 years ago.  At that time, it seemed like a good idea to use a fixed 10GB virtual disk drive for the XP C: drive in VirtualBox when I was building it.  Don’t ask why, it just seemed like a good idea at the time.

It turns out though that in the three years that I’ve been using this virtual disk image, successive software installs and upgrades have filled my virtual drive C: all the way up to the brim.  I noticed that my automatic SP2 upgrade to MS Office 2007 kept failing because there is not enough space left on the device.  I needed a bigger virtual disk.

After a fairly extensive search, I found what appeared to be just the ticket.  Someone else who had experience this problem figured out a fix and then took the time to write it up for others to use (thanks, Nick!)  Here’s the link, which describes nicely how to copy a virtual system disk over to a larger virtual system disk.

http://www.modhul.com/2008/10/21/re-sizing-a-virtualbox-virtual-disk-image-file/

Oh, and here’s the catch:

Remove any snapshots you may have – VirtualBox can go a bit wonky when re-sizing a VDI with snapshots attached (thanks to Darren for pointing this one out).

How’s that a catch, you ask?  Well, it turns out that there has been a bug or two in VirtualBox for years that can (usually does, apparently) prevent a user from deleting snapshot files in virtual machine.

Yes, I use snapshots, why wouldn’t I?  Snapshots allow you to roll back to a previous instantiation of your VM.  Let’s say your Windows VM gets a virus.  No problem: roll back to the snapshot you took a couple of weeks ago.  No more virus.  Now patch against that particular virus.

Bottom line:  I’m now unable to successfully copy my too-small virtual drive C: over to a larger disk, because the resultant copy is, indeed, wonky.

Now, let me make it perfectly clear:  I am not complaining about the maintainers of Sun (now Oracle) VirtualBox.  It’s free software, I’m grateful to have it.  It’s generally an excellent product. I’d like to point out from the perspective of being a developer and maintainer of free software myself (see this previous LJ article for a description of one of my FOSS packages):  producing and maintaining Open Source Software is a royal pain in the Kazoo.  You are always working with limited resources, and all you ever seem to hear is complaints.

Let me therefore say, “Thanks, VirtualBox developers and maintainers for getting us this far.” I'm sure you already have plenty of Kazoos to deal with. I’ll be happy to figure out a meatball surgery approach to get myself out of this latest Catch-22. And then I’ll write about the solution, too.

 

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

VBox drive size

Dale Finley's picture

Doug:

I am much happier with VBox when I use a very small drive. In order to do that with a Windows guest under Ubuntu I created a VBox shared file under my Home folder with sub-files for programs, documents, and every sort of file that might grow. There is no problem in installing windows programs on the shared file instead of the default c:/programs.

The machine seems just as fast this way, the windows disk doesn't grow, backups are simplified, and the wasted disk space that always seems to grow under VBox machine doesn't happen.VBox scares me when all my stuff is in the VBox folder, and five gigs is much more than you will ever need if you stop putting everything under your c: drive.

Dale

VirtualBox Potential

Joshua Rasnier's picture

Virtualbox has potential but I feel it is not yet close to being a enterprise solution.

virtual box

cameron's picture

It is good that these virtual technologies are moving forward though, it certainly keeps things more interesting and advancing.

Time Sheet & OOo Presentations, have your cake and eat it too!

Christopher M. Penalver's picture

Doug.Roberts,

Regarding the part of your article on time sheets:

"So, to fill out my time sheet in order to get paid, or to follow all the Track Changes in a Microsoft 2007 proposal document, I have to run Windows, because the company I work for uses 100% Windows software to run their business.
I accept that. And to ensure that I can fill out my time sheet, and track all those changes in MS Word documents, I run Windows in a virtual machine. (As an aside, I also use Open Office, have done so for years, but OO is not 100% compatible with MS Word. It’s also not always too swift with track changes..."

This type of compatibility issue regarding OOo & MS Office 200x, should be bugged with OOo ( http://qa.openoffice.org/issue_handling/submission_gateway.html ). If this is already done, please provide the link, your fellow OOo users are interested and appreciate it! :D

+ Regarding the part of the article on OOo presentation compatibility with MS Power Point:

"...and I cannot count the hours I have wasted trying to make an OO Presentation look like anything other than the dog’s dinner when viewed in MS Power Point."

If a compatibility issue is noticed then the same suggestion as above applies, bug it! One way I got around compatibility issues is to export the OOo Presentation to PDF and open that PDF document in MS Power Point. This avoids the problem while native OOo Presentation in MS Power Point bugs are hashed out.

Clone & install on a larger disk

Anonymous's picture

One other possibility is image your current system and unimage onto a larger drive.

Shutdown your VM. Boot the VM off from Clonezilla and image the C:.
Create a new VM with a larger C:. Install the image into the new VM using Clonezilla.

I've done a p2v conversion (of a Linux system) with Clonezilla to a smaller virtual drive then the original drive. YMMV

Thanks for the link

Nick Heppleston's picture

Doug, thanks for the link to my blog post - I was wondering why traffic was up!

Nick.

Small point I learned

Fat Pop Do Wop's picture

I recently decided to look into the issue of running out of virtual disk space, but I had the luxury of doing this BEFORE I had actually needed to, so wasn't under any pressure.

I simply added a second virtual hard disk (chose dynamically expanding) that was much bigger, booted to an image of Gparted, made a new NTFS partition, then copied all data over, removed the original disk (NOT deleted from Virtual Disk Manager), then booted to the new one. However it failed. PROBLEM was due to partition size. I had to re-partition the new virtual disk using an MS Windows XP installation disk image. This left a very small amount of free space at the end of the virtual disk but this matched whatever the MS Windows drivers were looking for. Then I booted to an Ubuntu Live image to copy the files and folders from the old drive to the new one, removed the old one again, then re-booted to the new disk and it all worked perfectly.

To this day I do not really know why the new partition had to be made using a Microsoft installation disk.

snapshots in VirtualBox

TheWall's picture

I, too, use VirtualBox for those 0.01%-of-the-time occasions when I have to use Windows for something. However, one thing I learned (after doing a lot of digging while working on an off-the-wall project) was that it's generally *not* a good idea to run with a snapshot of your VM all the time. The idea behind snapshots is that, when testing or upgrading, you have a roll-back point, and then when the test/upgrade/whatever is done, you remove the snapshot. The reason why is due to the way the snapshots work.

I haven't looked at the VirtualBox source code, so I could be a little off-base here, but from the testing I've done, it appears that when you take a snapshot of your guest OS, *only the changes to the guest file system* are written to your running disk image (vdi). In time, those changes get larger and larger, so you end up having a very large image file (the snapshot) and a second very large delta (the running image). This can get to be rather inefficient over a prolonged period of time, especially as you keep adding patches and updates to your Windows guest. Also, the longer you run a snapshot, the longer it takes to consolidate the changes into the running image or to revert to the snapshot, if there's a problem. Finally, if you lose the snapshot before you consolidate changes, you've lost your guest operating system, since only the changes get written to the running image (granted, though, that if the snapshot gets corrupted, you've probably got bigger problems, like the hard drive failing on the host operating system).

virus? In a VM?

Anonymous's picture

If you're really just using your XP image to enter your timesheet and edit work docs, you really shouldn't have much to fear in the way of viruses, no?

Naive, I guess - especially since my XP box at work had to be quarrantined recently.

How about using WINE to run MSOffice without a VM or Windows at all?

Not really

Doug.Roberts's picture

I had this old Win98 image running in VMWare about 6 years ago. I'd been using it for a couple of years to handle MS Word docs, not extensive usage at all.

Then one day I hooked up to the corporate network via a SecureClient ssh tunnel, and within an hour Bingo! A worm attack from some infected machine on the corporate net brought my Win98 system down. Had to reinstall it.

RE: Wine, I used to (try to) use Crossover's Wine implementation to run MS Office, but it was horrible, it constantly crashed. Maybe it's better now.

--Doug

Sir, Oracle's VirtualBox is

Anonymous's picture

Sir, Oracle's VirtualBox is NOT Free Software ! http://en.wikipedia.org/wiki/Free_software

Not Free?

Greg Bledsoe's picture

That's not correct. OSE (open source edition) which covers all desktop usage is GPLv2. Free/Libre and Free/Beer.

Some features (headless server operation) or covered by a PUEL (Personal Use and Evaluation License) which covers 99% of use cases and is Free/Beer though not Free/Libre.

I stand corrected

Doug.Roberts's picture

I'll send in my check to Oracle for $0.00 immediately.

--Doug

I believe Oracle only takes

admar@mailinator.com's picture

I believe Oracle only takes payment in souls these days ... you're $0.00 isn't going to cover that.

On a good note... Qemu/KVM is getting quite usable with "real" VM management UIs! Use something like that and avoid anything with the Oracle stamp like the plague is my take.

Thanks

Doug.Roberts's picture

That is good news about Qemu & KVM. I was going to play around with it once, but simply did not have the time. A usable UI will be most welcome.

Good advice about Oracle too.

--Doug

UI for Kvm & OpenVZ

Chris van de Wouw's picture

I recently started using proxmox. I had no xp with kvm or qemu whatsoever, but I was up and running within 15 minutes.

http://www.proxmox.com/

Easy solution

Floyd's picture

When faced with the same problem, I merely created a NTFS partition on my hard disk and shared it with samba. Move all of the documents, pictures, videos, etc. to that partition. Mapped it. Problem solved.

Windows Junctions

unregistered-Allan's picture

You should be able to create a "Program Files" directory on another disk and (after moving the original over) use junction.exe to make it appear as if it were on C:

It's possible with data files, and windows junctions are like hard links, but for directories.

http://msdn.microsoft.com/en-us/library/aa365006%28VS.85%29.aspx

Junction doesn't work across

Bob/Paul's picture

Junction doesn't work across partitions, does it? My understanding was they were basically hardlinks and had the same caveats that hardlinks on unix have.

Easiest solution, IMHO, is to create a second, larger virtual drive, boot from a LiveCD iso, then use gparted to copy/resize the partition from the small one to the new one. When you're done, remove the small drive from your virtual machine setup.

And comment for the OP: I delete snapshots all the time. What's this bug you're talking about? Works for me...

That's the most logical step

meanpt's picture

I agree with you as the solution to follow. In fact, this is what I preemptively do with my laptops - using an external disk to install non MS programs, in a directory called ProgramFiles2. The main and local disk is usually too small to cope with more than the updates for all the MS software which only installs in the main C: .

Thanks

Doug.Roberts's picture

That's good to know, I guess. It's also way more Windows than I ever care to learn. :)

--Doug

I thought about that

Doug.Roberts's picture

The reason this approach would not have worked for me is that it was Program Files that had filled up my C: drive. Those cannot be moved to another partition/drive, XP (or more accurately, my company's Altiris installer) insists that they reside on the C: drive.

Another similar approach would have been to create another virtual drive from within VirtualBox and then put an NTFS partition on it.

I do keep everything else that the XP guest needs to have access to (documents, mostly) on a VirtualBox shared folder, which is just another directory on my Linux host that I've told VB to grant the guest access to.

--Doug

Just create another Virtual Disk

Sachin Garg's picture

Just create another Virtual Disk - D: drive - and create another virtual disk and re-install Office in "D:\Program Files"
While you are at it, move your Documents to E: drive - problem solved.

I think you need to re-install Windows - 3 years without a re-install is way too long :)

Do this

Greg Bledsoe's picture

1. Make a new VM and a larger disk, but don't attach the disk yet.

2. Attach the disk to the existing windows vm.

3. Boot your windows vm from a live cd.

4. fdisk the new drive and create the partitions, etc

5. copy the data partition, dd if=/dev/sdold1 of=/dev/sdnew1

6. copy the mbr, dd if=/dev/sdold of=/dev/sdnew count=1 bs=446.

easy peasy

I did the same with Windows

pcero's picture

I did the same with Windows programs. First you make a new larger device (in my case from 10 to 60GB) and apply it to the VM as a second device. Then boot with a Hiren's boot ISO (or similar with clone software like Ghost) and clone the first over the second. Then eliminate in the VM the first device and you have the same system with 50GB more (or the GB you need). Test it and maintain the old and small copy as a buckup. Of course your technique is much better than mine, but at that time I had no idea of terminal possibilities in Linux.

easy fix

markh's picture

just make a new vbox hard drive mount both of them and boot clonezilla.....then use clonezilla to copy to the new vbox drive....problem solved.

I used that to copy a single xp install to make multiple virtual machines so it wouldnt give me the error of "a drive with checksum xxxxxx already exists etc"

Clever

Doug.Roberts's picture

Nice solution, thanks. What I did was to make a new VirtualBox install and copy the original c: hard drive file into it, which cleaned up all the snapshot mess. Then I was able to grow the file.

--Doug

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState