Improve software development department efficiency with VMware

Companies developing software for themselves or for customers know how complex, expensive and time consuming can be releasing a new product.

Development team members have to independently work on code and share it when needed, build and rebuild it on the same or different environments, while QA engineers have to test it on multiple configurations and scenarios, until the final deployment in production, where several factors are out of control and can mine stability and reliability.

IT managers always had small or no possibilities to mitigate technical issues and smooth the release path. But server virtualization changed everything becoming one of the first choices for boost the process.

In this article we’ll see how deploying a VMware virtual infrastructure can reduce most of the problems our development department encounter, speeding up its capability to deliver new products at new, unexpected levels.

VMware is not the only company offering virtualization solutions, but its wide range of products and its capability to seamless migrate a virtual machine from a platform to another makes it the best choice for this scenario.

Typical problems

The very first issue of a software development is environment integrity.

For many software engineers is normal having their development tools on the everyday workstation. When a new project starts the large majority of them simply start coding on the same environment they use for browsing the web, reading the email, watch videos or presentations, etc. Often even gaming on it.

Such systems should be perfectly clean, like the fresh installation where customers are supposed to host the product we are developing. Unfortunately this rarely happens.

Daily use for so many tasks imply a lot of installed software, which injects libraries, requires high-level permissions, modifies environmental variables and so on. Not to talk about possible malware infections.

Developed code could easily run or not run because of these variations but moving it on different machines, where operating systems have been compromised in different manners, will produce different results, leading to a complex and time-consuming troubleshooting.

Another frequent slowing down issue in complex projects is environment portability.

Software architects and engineers, product managers, have to verify how a product is growing during the whole development process or have to collaborate on it to improve or debug its routines.

Having many persons around the same monitor or permitting remote access to the development environment is highly unpractical. On the other way moving code from an operating system to another is not simple.

It’s not only depending on environment integrity, which cannot be granted in any way, but also a mere fact of delivering all parts needed to run a piece of code.

Any application based on database access, for example, is very hard to share if the developer, as often happens, has installed the database instance on his own machine or rely on a remote instance on a dedicated network segment where not everybody can access.

But even without a database, development team could be in need of libraries or, in case of multi-tiered applications, of other services which aren’t moved along with the code.

A third typical problem is the lack of physical resources.

When software engineers are savvy enough to not rely on locally installed services, they need remotely accessed services which have to be deployed, configured and notified to the team.

This requires time but most of all implies machines availability which cannot be given for granted.

In similar way often happens hardware configurations have to be modified during development, adding for example more memory or another network interface card.
Adding new components can be even more complex in big companies where hardware is acquired in stock from OEMs.

But the amount of machines for software development is not only limited to ones where to deploy needed services. It’s also depends on how many systems the company wants to support.

QA engineers which have to try the same code on several versions of the same operating system to verify our code works as expected in all possible scenarios: with different permission levels in several Windows editions or with different libraries availability in several Linux distributions.

A dedicated machine is expected to be available for each of them and things become very complex when multiple new applications are concurrently in development.

It’s worth to note that lack of hardware machines once solved for developers and testers can soon turn to be a problem for IT managers.

Once the big project is finished they have a certain amount of computers which will be wasted until the next one and could become obsolete in the meanwhile, obliging to replace some or all of them.

Even having enough resources, software engineers and testing staff still have to front the most frightening risk: time shortcoming.

A long series of logistical operations can severely slowdown development distracting coders from their focuses.

For example recognizing the need for environment integrity leads to the act of debugging code always on a fresh installation, which is impossible until developers reinstall the whole operating system from scratch every time.

But even without such level of attention, when the developed code includes an installer it’s critical working on a virgin OS.

Lack of time availability interests also testers, which not only need multiple physical machines for every platform where our code has to be certified as working, but also need to reinstall the same operating system several times, maybe because last installation failed or simply because have to test different languages, service packs or concurrent applications.

Fundamentally every test case should be conducted on a dedicated environment and this implies a notable effort. Even if disk backup solutions are in place they can help limitedly considering dependency on underlying hardware, which could change and require to save a whole new set of images, and restore times.

Improving the development phase

The most popular and oldest product from VMware is also the most important in the whole solution chain: Workstation.

Workstation offers a wide range of features able to address the largest part of mentioned problems in software development.

The probability a software engineer tries it and still sticks with traditional tools is near zero.

The first problem Workstation addresses is the one about environment integrity: developers and testers can count on the Snapshot feature which allows saving a virtual machine state and reverting back to it anytime is needed.

A savvy use of the snapshot feature implies developers install a brand new operating system in the virtual machine, fits it with all tools they need to produce new code and finally save a snapshot.

The operation grants a pure environment completely isolated from the every day workstation.

In this scenario a developer is able browse the internet, read his emails and even gaming on his own machine without jeopardizing the development workspace.

For maximum security the virtual machine could be completely disconnected from the real network card, so there are no chances the workspace can be compromised with a remote attack or virus infection, and there is no hassle to continuously patch the operating system or install an antivirus to maintain security.

But even if the workspace cannot be compromised it still can be overloaded with libraries, utilities and other things during a project.

In this case our software engineer can revert back to a clean state as soon as the project is closed simply calling back its first snapshot, within seconds.

Snapshot feature is pretty evolved and it’s a needful tool for QA as well.

When a compatibility testing is in progress testers need to assure the new application works correctly with several different products, from our company or third parties.

A snapshot manager permits to save multiple states of the same virtual machine, allowing testers to install one application after another without reinstalling the whole environment every time.

For example in a scenario where our new prototype application has to be tested for compatibility with Microsoft Office and several service packs, the best approach is to save a first snapshot of the just installed operating system, another after the non service packed version of Office has been installed, and still another one after the service pack is in place.

At this point testers are able to proceed with our code installation.

If something goes wrong or if they want to test the same installation with a different service pack, they just need to revert back to the snapshot taken before the service pack installation.

Trying to do the same thing without virtualization or a lot of different physical machines would take hours or days.

This process can be further improved thanks to other Workstation feature: multiple snapshots branches and linked clones.

Multiple snapshots branches feature permits to set an already taken snapshot as the original virtual machine image, and take new snapshots from there.

Linked clones act in similar way but disjoin the new snapshots from original virtual machine image location.

Both features are particularly useful for QA since they don’t copy what already exists of the original virtual machine but only refer to it for what will be done in future.

To better clarify we can reconsider the previous example: a tester in need of verify compatibility of a new application against multiple Microsoft Office versions and their service packs, can proceed creating a snapshot of the brand new operating system.

After installing Office 2003 over this snapshot the QA engineer will be able to set the new starting point on the snapshot he already took after installing the fresh OS.

At this point he will be able to invoke a new snapshot for both branches before install Service Pack 2 on Office 2003 and Service Pack 1 on Office 2000.

Our application can be tested against all these environments while the Snapshot Manager makes easy creating and discarding snapshots and linked clones.

Snapshots and linked clones not only drastically reduce time needed to prepare a new development or testing environment but also address another critical problem we already discussed: lack of physical resources.

With them QA engineers don’t need a new machine for every single environment to test, but just enough disk space to contain several branches of snapshots and clones.

Another great feature of Workstation is Teaming, useful both in development and testing.

Teaming allows logically linking together multiple virtual machines and launching them at the same time.

It also allows users to create virtual connections between them with simulated speeds.

So, for example, software engineer developing a multi-tier application can control how his code performs when used on a modem or a broadband connection, or usability tester can verify how much bandwidth is needed for a networked application to run without providing a bad user experience.


Addressing portability

As already said the biggest benefit of VMware software is capability to seamless share virtual machines between different products.

This not only permits developers to move without modifications and show their work to other team mates or product managers, but also allows to port applications to other virtualization facilities, where the code will be tested or even put in production.

So as simply as copying a folder the virtual machine containing the new software can be moved from Workstation to Server, the enterprise virtualization product which VMware offers for free.

There can be tested for compatibility, usability and its performances can be verified against stress tests.

After the QA phase then, the same virtual machine can be moved again in the VMware product aimed at datacenter deployment, ESX Server, where it will be put in production.

Anytime a problem appears the virtual machine can be moved back and forth between these platforms for patching errors or testing new configurations.

And if a customer wants an onsite demonstration of the new product, the same virtual machine can be moved once again, put in the VMware free virtualization product for desktops, Player, and distributed to sales managers worldwide.

Conclusion

Virtualization is not revolutioning just datacenter planning and deployment aspects. It’s also touching the most critical part of IT industry: application development.

VMware saw this before competitors and is creating a whole ecosystem improving software engineers’ efficiency by cutting away unproductive time.

As side benefits, companies fully adopting virtualization gain safer environments and flexible ways to reach new customers. But it’s just the beginning: today all operations are done manually but in a near future VMware will provide automation for some of them with a new product called Virtual Lab Manager which is expected before end of this year.

This will greatly simplify control and optimization of software production phases in big companies where multiple departments adopt different development tools but need to leverage virtual machines images in mandatory testing and production virtual infrastructures.

Automation is behind the corner. A new dimension in software development lifecycle too.

This article originally appeared on SearchServerVirtualization.


IBM to compete with VMware on virtualization benchmarking?

Quoting from the IBM official announcement:

IBM and Intel Corporation have joined in an initiative aimed at improving how IT managers select, deploy and measure virtualized server solutions for enterprise data centers.

One of the first tools to emerge from this joint initiative is a new virtualization benchmarking methodology called vConsolidate that runs multiple instances of consolidated database, mail, Web and JAVA workloads in multiple virtual CPU partitions on Intel-based System x servers to simulate real-world server performance in a typical environment. IBM and Intel are contributing the vConsolidate methodology to an industry standards body for consideration.

Using vConsolidate to benchmark the IBM System x3950 server with four dual-core Intel® Xeon® 7100 processors shows the x3950 delivers up to 46 percent more performance throughput than a competing system when running a mix of larger two- and four virtualized processor partitions.

Based on this and other customer test results, IBM and Intel created a VMware Infrastructure Sizing Guide aimed at helping customers select and appropriately configure the various virtualized server options available to them.

The result is a tool that provides recommendations for target utilization rates, the total number of virtual machines that will be needed to run the application, and the number of physical servers required to support the computing workload and goals.

To assist customers with making virtualization adoption decisions, IBM expects to open the Virtualization Resource Center (VRC) in early 2007. Customers will be able to apply principles gleaned from vConsolidate and sizing guide activities to their particular environments and software workloads…

VMware is developing a benchmark model for virtualization platforms (VMmark) since a lot, and first details about company approach were disclosed at VMworld 2006.

VMware hopes VMmark would be widely accepted by other virtualization vendors and major OEMs, and it’s working with SPEC to standardize it.

Description of IBM vConsolidate is pretty similar to VMware VMmark in approach and aims to standardization. Also IBM is SPEC partner as well as VMware.
Is IBM taking distances from current VMmark development path?

Tech: How to generate a memory dump file on Virtual Server 2005

Microsoft published an interesting support article for developers trying to generate memory dump file inside Virtual Server virtual machines:

Microsoft Windows operating systems let you use the keyboard to generate a memory dump (Memory.dmp) file. When this feature is enabled, you can generate the Memory.dmp file by holding CTRL on the right side of the spacebar while you press SCROLL LOCK two times. This step-by-step article describes how to use this feature when you run a Windows operating system on a guest computer in Microsoft Virtual Server 2005.

Read the whole article at source.

Thanks to Andrew Dugdell for the news.

Book: Professional VMware Server

Wrox published the first book dedicated to the free enterprise virtualization product VMware Server (formerly GSX Server):

Developing software and testing within virtual machines provides you with countless advantages in terms of automation, stability, disaster recovery, and overall code quality. With this book as your guide, you get the tools you need to create a fleet of virtual machine images that you can use to build your own secure applications.

Beginning with the basics of virtualization and its place within the development and testing environment, the book then delves into the APIs and provides a deep, example-rich reference to their function and purpose. VMware expert Eric Hammersley shares his vast experience in using VMware in the development process and shows you how you can streamline your own development with the new VMware Server.

You’ll learn to integrate VMware Server into your existing environment using CruiseControl.NET and Visual Studio® Team System as examples. Plus, you’ll get a clear understanding of virtual hardware, networks, and the server console, and find out how to build base images. You’ll discover the benefits of software development and testing within virtual machines and the advantages they have to offer. This will help you rapidly deploy new machines, set up multiple platforms on a single machine, and dramatically increase the utilization of your hardware.

What you will learn from this book

  • Steps on how to install and configure VMware Server
  • Tips to follow when creating development base images
  • Methods on how to organize your image libraries and prepare them for use
  • Best practices for using VmCOM, VmPerl, and the Programming API
  • How to integrate VMware Server into your existing environment and automate its functionality
  • Ways to leverage VMware Server to provide banks of testing machines on demand

See book details and reviews at the virtualization.info Bookstore.

Microsoft VHD Test Drive program collects notable interest

Tony Bailey, Senior Product Manager at Microsoft, discloses the VHD Test Drive program launched in November achieved 300,000 downloads of VHDs.

Microsoft is expected to extend the program before end of year, publishing more of its backend servers as well as other Windows-based virtual machines charged with partners solutions.

A great help for customers would be releasing a WSUS virtual appliance alread updated with latest Windows patches.

Whitepaper: Integrated Virtualization Manager on IBM System p5

IBM RedBook department released a new whitepaper (actually a Redpaper) about virtualization management features included in System p5 servers:

The IBM Virtual I/O Server Version 1.2 provided a hardware management function called the Integrated Virtualization Manager (IVM). It handled the partition configuration on selected IBM System p5, IBM p5, and IBM OpenPower systems without the need for dedicated hardware, such as a Hardware Management Console. The latest version of VIOS, 1.3.0.0, adds a number of new functions, such as support for dynamic logical partitioning for memory and processors in managed systems, task manager monitor for long-running tasks, security additions such as viosecure and firewall, and other improvements.

The Integrated Virtualization Manager enables a more cost-effective solution for consolidation of multiple partitions onto a single server. With its intuitive, browser-based interface, the Integrated Virtualization Manager is easy to use and significantly reduces the time and effort required to manage virtual devices and partitions.

This IBM Redpaper provides an introduction to the Integrated Virtualization Manager, describing its architecture and showing how to install and configure a partitioned server using its capabilities.

Read the whole whitepaper at source.

Tool: VMDK to VHD Converter

The new website vmToolkit released its first handly tool: a converter for Windows able to translate a VMware virtual machine format (.vmdk) in a Microsoft virtual machine one (.vhd).

The opposite migration is already possible thanks to VMware Workstation native capability to import Virtual PC and Virtual Server images, but also thanks to VMware Importer 2.0 and upcoming VMware Converter 3.0 free tools.

So far Microsoft never released a tool for achieve similar migration and customer had to rely on commercial virtual to virtual (V2V) solutions.
Now worldwide Microsoft customers using Virtual Server can convert for free hundreds of VMware virtual appliances.

Andrew Dugdell already published a positive comment on it:

The VMDK was 1,986 Mb and it converted to a 2,001 Mb VHD in about 15 minutes on my aging T41p laptop.

Everything is working nicely, I have a couple of errors in the VM eventlog – but humming along smoothly.

Download it here.

rBuilder supports multiple virtualization platforms

rBuilder is the innovative platform offered by rPath able to simplify and partially automate creation of Linux-based appliances.

Since ever rBuilder supports (live and installable) ISOs and VMware .vmdk output formats, becoming one of the preferred solution to develop new virtual appliances. But if you are not using VMware products you had to convert these images in a more flexible format, with a so-called virtual to virtual (V2V) migration.

Now rBuilder allows users to save their distros in raw hard disk image format, permitting to use them in other virtualization platforms like Xen, Parallels or QEMU.
And, not being enoug, rPath is now extending support to Microsoft .vhd format, allowing everybody to create virtual appliances for current Virtual Server 2005 R2 and upcoming Windows Server Virtualization.

Parallels Desktop achieves Macworld Editor’s Choice Award

Macworld published its annual list of preferred software, hardware and online products for this 2006. Among 29 winners of Editors’ Choice Award appear Parallels, which conquered Apple customers with its Desktop virtualization product.

At the moment Parallels has no competitors on virtualization market for Apple: Microsoft retired its Virtual PC for Mac OS and VMware still has to deliver something concrete after its announcement, but customers want more. In particular they are demanding for 3D support in virtual machines, and Parallels may announce something in this direction at imminent MacWorld 2007 in January.

Altiris extends Server Management Suite support to VMware products

Quoting from the Altiris official announcement:

Altiris, Inc., a pioneer of service-oriented management solutions, today announced the addition of new Linux patch and virtual server management capabilities to help secure and manage heterogeneous server environments.

Altiris provides a suite of server management solutions to help improve system availability and reliability, monitor performance, maintain operations and remediate security vulnerabilities. New features will include:

  • Improved support of VMware virtual server resources now includes expanded deployment, inventory and software delivery for VMware ESX Server and GSX Server


Altiris® Server Management Suite software is now available starting at an MSRP of $253 USD per node with volume discounts available…