Moose Logic and DataCore^TM

Consolidating like or unlike disk resources.
Virtual Disk Pooling is the over arching feature responsible for consolidating storage capacity from like or unlike disk resources. As noted earlier, the pool may encompass a variety of brands and models of disks, effectively creating storage tiers with different price/performance/capacity characteristics.

Pooling is fundamental to storage virtualization capability, enabling virtual (or logical) disks to be rapidly created from blocks of space on the physical devices. Using a central administrative interface, these virtual disks can then be assigned to storage consumers throughout the SAN with specific access permissions, possibly shared among different servers, virtual machines or clustered applications. The upper limit on a SANmelody storage pool is 32 Terabytes, whereas SANsymphony scales well into the petabytes.

Below you see how a DataCore storage pool can be composed of different brands and models of disks and managed as a central resource to meet the varied needs of applications. Before we exhaust the capacity of some devices in a specific tier, we can add more, possibly using the next generation product from the same supplier or a new brand that offers us the same reliability with more favorable price/performance. And as new storage consumers arrive they can immediately tap into the storage pool via their iSCSI or Fibre Channel SAN connection.

Real-time I/O replication for High-Availability
When it comes to non-stop access, synchronous mirroring deserves most of the credit. It handles the real-time replication of I/Os for the ultimate in high-availability SANs. Having two or more nodes storing the data simultaneously in conjunction with multipath I/O drivers on the client computers eliminates the DataCore storage pool as a single point of failure.

SANmelody allows you to configure redundant storage pools by synchronously mirroring between two DataCore nodes. SANsymphony scales to accommodate N+1 redundant nodes. Meaning that if you need 4 nodes to handle the I/O workload, you add a fifth node to absorb the added load when any one of them is taken out of service. During normal operations all five nodes would be fulfilling part of the work but each would have enough headroom to cover a portion of a nodal outage.

For any mirrored virtual disk, one DataCore node owns the primary copy and another holds the secondary copy. They maintain these copies in lock step by synchronously mirroring updates in real-time from the primary to the secondary.

In the diagram below, Node A owns the primary mirror labeled “P” and Node “B” holds the secondary copy labeled “S” for a given virtual disk. The preferred path from the client computer to the virtual disk is assigned to the node that holds the primary copy of the mirrored set.

Under normal operation, all read and write requests issued to that virtual disk will be serviced by the primary copy. The secondary copy need only keep up with new updates arriving from the mirroring function. Generally, nodes are configured to control primary copies for some virtual disks and secondary for others, thereby evenly balancing their read workloads.

Should any errors be encountered on the preferred path, the client’s multipath I/O drivers automatically fail over to the alternate path without disrupting applications. The same is true if a node needs to be taken out-of-service for maintenance or upgrades.

From a physical standpoint, best practices call for the DataCore nodes be maintained in separate chassis at different locations with their respective portion of the disk pool so that each can benefit from separate power, cooling and uninterruptible power supplies (UPS). The physical separation reduces the possibility that a single mishap will affect both members of the mirrored set. Round-trip network latencies govern the maximum distance between mirrored nodes. Current technologies support inter-node distances up to 35 kilometers using Fibre Channel.

Allocate just enough space, just in time.
In these tougher economic times, thin provisioning gets a lot of press and rightly so. DataCore’s implementation spans any physical devices in the pool that you choose and like its other features, is not dependent on any one host, operating system or hypervisor.

There are two parts to thin provisioning. On the one hand, the storage consumer is fooled into thinking that it has very large drives all to itself. In reality, the volume only takes up physical space when it is actually written to. This solves the problem of guessing wrong and wasting space when an application first asks for a lot of capacity even though it may never use it.

Rather than tie up all that space, we hand out blocks only as needed, just in time. This works out well with virtual machines that like to mount volumes that are much bigger than they actually need. Some refer to it as "oversubscription."

In the example below, we’ve logically assigned 6 TBs, but the applications have only written to less than one-half TB. When real consumption starts to exhaust our physical space, we alert the administrator to add more. You get to set when the alarm goes off. Once more drives arrive, you instruct the software to add the new capacity to the existing pool to give you more breathing room. All that can happen non-disruptively. Moreover, the additional space could come from completely different equipment given your preferences at the time. DataCore also includes utilities to reclaim space temporarily consumed on thinly provisioned virtual disks and later zeroed out by the client computers.

Speeds up performance.
High-speed caching has long been a potent differentiator for DataCore’s products. If you have been conditioned by other suppliers to believe that any form of virtualization brings a performance penalty, think again. In the process of virtualizing disks, DataCore software accelerates reads and writes by leveraging the powerful processors and large memories of the x86/X64 servers on which it runs.

Up to 1 Terabyte of cache may be configured on a DataCore node enabling it to perform at solid state disk speed without the expense. Caching essentially recognizes I/O patterns to anticipate which blocks to read next into RAM from the backend disks. That way the next request can be fulfilled quickly from memory at electronic speeds.

When computers write to a virtual disk, the information goes quickly into memory and later de-staged to disk, often grouped with other writes to minimize delays when storing the information to disk.

The simplest way to look at caching on a DataCore node is to view it as a level 1 cache that can respond in less than 20 microseconds, whereas the caches on the disk array would be slower and pricier, taking somewhere in the hundreds of microseconds. Both of these caches aim to hide the much longer delay of the physical disk I/O which is in the order of 4000 to 6000 microseconds (or 4 to 6 milliseconds).

In the diagram you see how the software uses the CPUs on the DataCore nodes to rapidly poll for inputs and immediately service input or output requests out of SAN-wide caches. Polling replaces slower (higher latency) interrupt servicing techniques that take much longer to recognize I/O events. The quicker the CPU, the faster we can turn around an I/O. The same polling method is used whether fielding requests from the SAN clients or responding to the back-end disks. Write coalescing reduces the delay in writing to disk by grouping inputs, whereas pre-fetching into cache anticipates blocks to be read given earlier reads from that section of the disk.

Better protection and performance.
RAID (or Redundant Arrays of Inexpensive Disks) is a common way to gain better performance and protection by spreading I/O across multiple disk spindles. Virtual disks may be intentionally striped across several physical drives or simply mapped to a logical RAID drive supplied by the underlying disk subsystems.

Thin provisioning takes advantage of RAID striping to dynamically allocate more disk space when the initial set of drives runs out of room.

Improve response and throughput
Load balancing across the back-end channels into the physical storage pool complements caching to improve response and throughput. The storage consumers on the SAN may also be doing host-based load balancing across their ports.

Load balancing helps to overcome short-term bottlenecks that may develop when the queue to a given disk channel is overly taxed, or when one channel fails or is taken offline.

Transparently move contents from one disk to another
Another by-product of making storage interchangeable using DataCore’s storage virtualization software is the ease with which you can relocate data from one physical disk to another, non-disruptively. We refer to it as virtual disk migration or volume replacement.

It comes in handy during hardware upgrades as you seamlessly roll in new equipment in place of older hardware. Having copied the contents to the new drives in the background, the software will zero out the original disk and reclaim its space into the free pool.

The more visual example below shows how easily you can replace storage devices in the background while applications continue to run uninterrupted. Sometimes the older equipment can be put to good use as a lower tier device for less demanding requirements, or it can be decommissioned altogether. The choice is yours.

DataCore also excels at migrating operating system images and data between physical and virtual machines (P-V) and between unlike virtual machines (V-V). The “Transporter option” rapidly imports and converts physical and virtual machine disk images encoded with a different format. It significantly shortens and simplifies transitions between dissimilar platforms. The migration is quick and verifiable at the outset while the bulk disk-to-disk copy occurs in the background.

We never modify the original disk images on the source machine during the conversion to avoid irreversible migrations.

Capture point-in-time images quickly
Once you’ve tried online snapshots, you can’t live without them. Snapshots capture a known good point-in-time that may be used for several purposes without scheduling lengthy back-up windows. It may give you a recovery point to undo a patch or file deletion. Or it may be used to feed business intelligence analysis. They are also commonly used to verify new software enhancements in test and development before being put into production.

Snapshots are invaluable in cloning working system images to provision identical new servers. Although snapshots utilities are commonplace in operating systems, hypervisors, backup software, and disk arrays, capturing them at the SAN level affords some major advantages. For one, there is no dependency on host software. Nor does it consume host resources. And you don’t need mutually compatible disk arrays. You can snap the contents of disks on a tier one array and place it on a tier 2 or tier 3 device, rather than tie up expensive space on the top-of-the-line equipment.

In fact, DataCore snapshots reduce capacity consumption in two ways:

1. They can be set to capture only changes from the original source, and
2. They may reside on thinly-provisioned volumes.

DataCore snapshots may be triggered from Windows Virtual Shadow Copy (VSS-compliant) applications that temporarily quiesce the application and flush O.S. caches to ensure a consistent recovery image.

There’s some impressive magic that goes into making DataCore snapshots fast and lean. Part of it comes from the realization that generally, only a few blocks on most devices really experience much change. With that in mind, the snapshot can reference the original unchanged blocks most of the time without taking up any extra room. We need only keep a separate, unmodified copy of those blocks that get updated. And that copy is taken on the first change to that block, effectively giving us the frozen point-in-time. Think of the software as a traffic cop. If the block is unchanged, read from the source. If the block has been changed, read from the copy of the original contents kept in a separate bucket.

DataCore gives you several snapshot variations. We just described the “Snapshot Image”. You can also ask it to completely clone the source disk in the background. We refer to it as a “Complete Image Copy.” You may refresh the snapshot to a later point-in-time using the “Image Update” command, or you can completely replace the source with an earlier snapshot using the “Source Update” command. That one is reserved for those cases when you have to undo changes.

Maintain distant copies up-to-date without impacting local performance
Earlier, we discussed how synchronous mirroring fits into our solution for high-availability. We spoke of an upper limit near 35 kilometers between mirrored nodes before round-trip latencies make lockstep replication impractical today. In other words, if you tried to synchronously mirror at longer distances, applications might time out waiting for the acknowledgement to come back from the remote end.

DataCore’s remote replication function, internally called “Asynchronous IP Mirroring” or AIM, addresses requirements for secondary copies to be housed beyond the reach of synchronous mirroring, as in distant disaster recovery sites.

It relies on a basic IP connection between locations and work in both directions. That is, each site can act as the disaster recovery facility for the other.

The software operates asynchronously, meaning that it does not hold up the application waiting on confirmation from the remote end that the update has been stored in both places. Instead, it offers to do its best to keep up to date with changes at the local site, but makes no guarantees. It’s far better than trying to constantly make backup tapes and ship them to a safe house or paying extra for point-products to handle only this task.

Control and monitor all SAN resources from one console
System administrators control and monitor DataCore functions from one central console via an intuitive user interface. The same tools are used across like or unlike devices. SANsymphony offers a richer, more comprehensive administrative experience that spans the entire SAN. It also automates certain tasks that would require manual intervention in other products.

Special plug-ins are also available for integration into server virtualization consoles.

Return to any point in time without taking explicit backups
some organizations must restore their environments to specific points-in-time that may not coincide with the scheduling of their regular snapshots or backups. Nor can they afford to interrupt or quiesce applications so frequently. Continuous Data Protection (or CDP) is for them.

This is a separately-priced SANsymphony option performed on a separate DataCore node. As the name implies, it continuously logs and timestamps I/Os written to designated virtual disks allowing you to restore the environment to any arbitrary point-in-time within that log.

DataCore calls this feature “Traveller^TM” as in time traveler. CDP automatically combines periodic snapshots with a log of time-stamped updates to roll back the image to an earlier point. The administrator can dial back to this prior state via an intuitive user interface. In this example, we have a rolling log covering the last 5 days of updates from which to fall back to.

Spread DR responsibility across multiple smaller sites
ASR is a key component of DataCore’s comprehensive business continuity portfolio. It automates and radically simplifies how one or more, smaller, remote IT facilities take over workloads from a central site in the event of a disaster or scheduled outage. Rather than attempt to fully recreate the central datacenter at another major site, ASR can distribute responsibilities for keeping the business going among a few remote offices and branch offices (ROBO) based on their computational, networking and storage capacities.

ASR also takes care of returning control of the workloads to the central IT site when the main IT center is deemed capable of accepting them.

Unlike other approaches that can only address a subset of the IT environment, DataCore covers both virtual and physical servers without taxing applications, hosts, or hypervisors. Nor does ASR depend on duplicating expensive equipment offsite, such as top-of-the-line disk arrays and specialized networking gear. Only DataCore allows organizations to leverage readily available IT assets, often differing between sites, to minimize or eliminate the disruption and data loss attributed to planned and unforeseen site outages. In view of the fragmented and complicated alternatives, DataCore offers a refreshingly rational and cost-effective solution.

ASR builds on the virtual disk provisioning, asynchronous remote replication and online snapshot features to constantly update remote sites with changes occurring at the central datacenter.

Tracks and charts long-term SAN behavior
The built-in administrative interface in both SANmelody and SANsymphony provides real-time status on the virtual storage pool. DataCore also offers optional analysis and reporting tools (SANmaestro) to track and chart historical behavior over extended periods. Such charts reveal trends in resource consumption and performance that can be used for tuning, troubleshooting and capacity planning. The information may be synthesized and exported to third-party packages for further analysis and billing.