Your are here: Home > Blog

This is the first of two videos addressing virtual storage and its benefits. There are a number of storage solutions out there on the market but we have chosen to focus on DataCore of the purposes of this video. DataCore is an iSCSI SAN solution and you can learn more about their products here.

In part one, we address thin provisioning and virtual volumes. Watching this video will help you understand part 2 of “What is Storage Virtualization” where we talk about how multipathing relates to virtual volumes and contributes to a highly available SAN solution.

Yesterday (August 25), Citrix formally announced XenDesktop 4 Feature Pack 2. It’s expected to be available by the end of September, and, of course, will be available at no charge to existing XenDesktop customers whose Subscription Advantage is current. The big news in this Feature Pack is the incorporation of XenClient and XenVault.

We’ve talked a lot about XenClient here, but haven’t said much about XenVault. It’s high time we did, because it’s a pretty cool piece of technology in its own right.

If you’ve used Citrix products in the past, you know that we have administrative control over whether, for example, users who are running applications on a XenApp server are able to save data back to a disk drive on their client device. With the advent of Smart Access (enabled by Access Gateway Enterprise policies), we can get even more granular: we might allow a user to save data to a client drive if they’re connecting from within the protected network, or connecting from a corporate-owned laptop, but deny that same user the ability to do so if they’re connecting from a personal device or public location like a hotel business center.

Unfortunately, once the data is on a client device, you now have a security risk. It could potentially be copied to a USB drive. The corporate laptop could be lost or stolen. (For some of the more high-profile examples, check out the “laptop losers hall of shame.”) Nevertheless, it’s often viewed as a risk we have to take so that our mobile users can be productive.

XenVault, which was first previewed at the Synergy event last May, is designed to address this risk. XenVault is a new plug-in for the Citrix Receiver. As such, its deployment and configuration are controlled through the Citrix Merchandising Server. To quickly review, Merchandising Server is the preferred tool Citrix has provided for installing and configuring client software. The first time a user authenticates to the Merchandising Server (through a simple browser interface), the Citrix Receiver will be pushed down and installed on the client device, together with whatever plug-ins and configuration details the administrator has defined for that user. Subsequently, the Citrix Receiver will check back with the Merchandising Server behind the scenes, and receive any configuration updates that may be available.

The XenVault plug-in creates a secure, encrypted (256-bit AES) storage area on the client hard disk. Typically, any application that is running remotely on a XenApp server or XenDesktop virtual PC will only be able to store data in the secure, encrypted location, if it is allowed to store data on the client drive at all. Same for an application that has been streamed via XenApp for local execution on the client (regardless of whether it was packaged with the Citrix streaming tools or with App-V). While the user will be able to use Windows Explorer to look at the secure location and see what files are there, the user will not be able to copy files from the secure location to a non-secured area of the hard disk, nor open the files with applications other than those specified by the administrator. For a deeper explanation of how this works, see Joe Nord’s blog post on the subject.

If the laptop is lost or stolen, the administrator can issue a “kill pill” that will cause the secure, encrypted area to be locked or deleted the next time the Receiver checks in with the Merchandising Server. Pretty cool.

If you can’t wait until the end of September to try it out, and you have a mycitrix login, you can download the XenVault technology preview now. And keep watching this space, because I’ve got a feeling that this will be a good subject for a future video blog.

Recently Steve Parlee, Moose Logic’s Director of Engineering, sat down with Tim Warden, DataCore’s Western Region Director of Sales. Moose Logic has installed a number of DataCore solutions over the last few years and highly recommends their software to anyone looking into storage virtualization. We’ve also mentioned DataCore a number of times in our blog and newsletters. If your still not sure what DataCore does, this is a great introduction to their storage solutions. In the interview, Tim Warden explains the benefits of the DataCore software and what their solution can bring to your data center.

Last fall, we posted about Citrix Provisioning Services and Microsoft KMS activation. To briefly recap, here’s the issue:

  • When you convert a Windows 7 OS image to a shared image for provisioning, it breaks the Microsoft license key.
  • The way you deal with that is to use Microsoft’s Key Management Services (KMS) to auto-activate systems as they boot.
  • A KMS server must have a minimum number of systems checking in for activation before it will activate anything (5 different server systems must check in before it will begin activating servers, and an aggregate of 25 servers and/or workstations must check in before it will begin activating workstations.)
  • If your KMS server is running on Windows Server 2008 R2, both physical and virtual systems will increment the counter. If it’s running on an earlier server version, only physical systems will increment the counter.

In the comment thread of that earlier post, “Chris” stated that he was trying to use Provisioning Server to provision Windows 7 systems, but that they were not incrementing the counter on the KMS server. It turns out that he was absolutely right, and I thought this was important enough to bump the issue by writing another post rather than just going back and commenting on the older one.

It turns out that, although Provisioning Server changes the host name as systems boot, it does not change the machine ID (“CMID”). And, unfortunately, the CMID is what a KMS server looks at to determine whether a machine that’s checking in is a new one that hasn’t previously checked in. Therefore, all of your provisioned Windows 7 systems will look to the KMS server like the same system checking in over and over again, and will not continue to increment the threshold counter.

According to a blog post by Thomas Koetzing a couple of weeks ago, Citrix has told him that this will be fixed in the next release of Provisioning Services, scheduled for sometime in Q4.

Frankly, I’m pretty disappointed by this whole issue. Windows 7 has been out now for almost a year. The big push by both Citrix and Microsoft is that XenDesktop is a great way to roll out Windows 7. Provisioning Services is a must for any significant VDI deployment, because otherwise you eat up far too much of your expensive SAN storage. But yet we’re still stuck in a situation where we can’t use Provisioning Services to provision Windows 7 unless we have at least 25 physical systems checking in with our KMS server for activation. In my opinion, there is no excuse for this issue not being addressed long ago…particularly when it’s been a known issue since the release of Windows Vista.

I did find a workaround described by Kirk Kosinski in a Citrix forum post:

What I did was create a VM with VL media, sysprep and power off, convert to a template, then deploy the template 25 times and boot each VM once (a few required a reboot before contacting the KMS for whatever reason). My KMS server could then activate clients successfully, at least for a while… the activation count will decrease over time if the machine doesn’t contact the KMS server, so you will periodically need to redo this process.

The VMs don’t have to join the domain to activate so you don’t need a complicated sysprep script, just make sure to not include any license key in the script…

This strikes me as a bit of a pain, particularly when you’ve got to do it every six months or so to keep your systems alive, but it should at least work until Citrix and Microsoft get this sorted out.

We have made a number of posts on this blog discussing the value of Citrix XenDesktop and felt it was time to add a video to this topic. Sid Herron had written a previous post Minimum Requirements for XenDesktop that you might find helpful after watching this video. Between this video and Sid’s post you should have a basic idea of what you would need for a basic deployment of Citrix XenDesktop.

Take a few minutes to learn what is required and what would be optional in a XenDesktop deployment, as well as how all the pieces would interact.

Over the past few months, we’ve made several posts about XenClient. But in case you haven’t read them, or you need to refresh your memory, XenClient is (quoting from Citrix here): “…a high-performance, bare-metal hypervisor that runs directly on the client device hardware, dividing up the resources of the machine and enabling multiple operating systems to run side by side in complete isolation.”

Of course, there are other ways to run multiple operating systems side by side on a client device, although they may not give you the level of performance that XenClient – because of its small footprint – brings to the table. The tricky part is figuring out how to manage that environment once the user unplugs the laptop from the network and takes it on the road. How do you patch it? How do you back up user data? What do you do if the laptop is lost or stolen? If one of the OS instances is corrupted, or accidentally deleted, how do you get it back?

That’s the job of the Citrix Synchronizer – a virtual appliance that runs back in your data center and communicates with your XenClient-equipped laptops securely (via SSL) over the Internet. But rather than try to describe to you in detail exactly how that all works, it’s probably easier to simply show you. So take a few minutes to watch our own Steve Parlee demonstrate the interaction between Synchronizer and XenClient.

And Just to Prove the Point…

August 5th, 2010 | Posted by Sid Herron in General | Security - (0 Comments)

Monday, I wrote a post about some of the latest trends in cyber crime.

Tuesday afternoon, our Web site was hacked.

We didn’t realize it until we landed on the Google blacklist this morning, although I should have suspected something when I noticed, on Tuesday afternoon, that both of our two instances of WordPress – the one that powers this blog, and the one that powers our “News” page – had stopped working. But, since I knew that I was a couple of revisions behind, I elected to upgrade my WordPress instances to the latest release. When they came back up working again, I didn’t probe any deeper. I should have known better.

Log analysis indicates that our FTP account was compromised. Beginning at about 3:18 pm PDT on Tuesday afternoon, a series of files were uploaded to our server from an IP address that appears to be located somewhere in the UK (in the London area, to be more precise). The file transfers were done using the FTP account for our domain. They went through our site and changed every index.* page. Specifically, they placed a “hidden iframe” immediately following the <body> tag.

For those who aren’t conversant with HTML, you can think of an “iframe” as a window on a Web page that displays content from another Web page. Except that, in this case, the height, width, and border width of that window were set to “0.” The point being that when your browser loaded the page from our site, it would also load the content from the other site, but it wouldn’t be visible on the page. That content was, no doubt, some kind of malware that was intended to do something bad to your system. The hidden iframe attack is one of the most common exploits out there, and is typically used for some kind of “drive by” malware distribution campaign where the bad guys try to place their hidden iframe on as many legitimate sites as possible. When you visit the site, your browser fetches the code, and now it’s a matter of how good the defenses are on your PC.

Obviously, we’ve changed the FTP account credentials. But, frankly, we’re still not sure how the account was compromised in the first place. It was a pretty strong password, and not one that you’d expect to fall victim to a dictionary attack. We’ve been running malware scans on the machines that we normally use when we work on the Web site, and have yet to come up with a “smoking gun” that would explain how the credentials were compromised.

So…what to take away from this? First of all, it’s no fun to become a statistic. Second, nobody is immune to this sort of thing. Even the CBS News Web site was hit by an iframe attack not that long ago. Nobody is too big or too small to be targeted. Third, change your passwords regularly, even if you think you have strong ones. Fourth, be suspicious when something unusual happens. I should have dug deeper Tuesday afternoon, but it was late in the day when it happened, and I settled for what looked like an easy fix. Finally, it’s a pain in the you-know-what to go through and clean up the aftermath of something like this. It’s cost me most of today, plus we’ve been on the Google blacklist all day and probably won’t come off of it until sometime tomorrow when they’ve had time to re-scan our site.

The bad guys are out there, and they do want your stuff. Be careful.

Causes and Costs of Cyber Crime

August 2nd, 2010 | Posted by Sid Herron in Security - (0 Comments)

I read a couple of items today about security and cyber crime that I found rather interesting. One was an article that came out a week ago on infoworld.com about the “First Annual Cost of Cyber Crime Study,” conducted by Ponemon Institute. The study involved 45 midsize and large organizations, ranging in size from 500 to more than 105,000 employees. They represented a mixture of industries and government agencies. The study revealed that cyber crime cost these organizations an average of $3.8 million dollars per year…each. The reported costs ranged from a low of $1 million to a high of $52 million per year.

The reported costs represent the direct cost of coping with attacks, including such things as, for example, the amortized annual cost of a Web application firewall purchased to respond to an attack on a Web application. They also included the time spent responding to attacks, the cost of disruption of business operations, lost revenue, and the destruction of assets. They found that it took an average of 14 days to respond to a successful cyber attack, at an average cost of over $17,000 per day.

Admittedly, a sample size of 45 companies is relatively small. But still – $3.8 million per year, average? Holy smoke!

The other piece of light reading will help to flesh out the picture and add some perspective. It’s the 2010 Data Breach Investigations Report conducted by the Verizon RISK team, in cooperation with the U.S. Secret Service. It combines data from Verizon’s 2009 case load with additional data contributed by the USSS to form a data set that spans six years and over 900 security breaches, representing over 900 million compromised records. About two-thirds of the breaches covered in the report have either not yet been disclosed, or never will be.

While the cases worked by the USSS more frequently involved insiders, in Verizon’s own cases, almost all data stolen in 2009 – 98% – was the work of criminals outside the victim organization. 85% of that data was stolen by “organized criminal groups.” For a definition of “organized criminal groups,” see Appendix A of the report…it’s pretty interesting reading in and of itself.

Not surprisingly, financial services organizations were most frequently targeted (33% of cases), for the same reason Willie Sutton robbed banks: that’s where the money is. But you may be surprised to learn that the hospitality industry wasn’t that far behind (23% of cases), followed by retail (15% of cases). And here are some other things that might surprise you (note that the following percentages add up to more than 100%, meaning that some cases involved more than one factor):

  • 48% of breaches involved “privilege misuse” (that’s up 26% from the year before). The report defines this as any use of resources or privileges in a manner contrary to that which was intended, whether malicious or non-malicious. This category includes obvious actions such as embezzlement or deliberate theft of information by an insider, but also losses that resulted from abuse of system access, use of unapproved devices, violations of an organization’s Web or Internet use policy, abuse of private knowledge, use of unapproved software or services, unapproved changes and workarounds, and violations of an organization’s asset / data disposal policy.
  • 40% resulted from hacking (down 24%) – the majority of which involved either the use of stolen login credentials, or SQL Injection attacks. A fair number also involved exploitation of default or guessable credentials (or cases where no credentials were required), and brute force and dictionary attacks.
  • 38% utilized malware (unchanged)
  • 28% employed “social tactics” (up 16%) – using deception (spoofing, phishing, forgery), manipulation, intimidation, bribery, extortion, etc., as a means of breaching an organization’s security. Social tactics are often combined with other categories, for example, malware designed to look like antivirus software.
  • 15% were physical attacks such as theft, tampering, and surveillance (up 6%)
  • And what may be the most astounding finding of all: “…there wasn’t a single confirmed intrusion that exploited a patchable vulnerability.” Does that mean you don’t have to pay attention to patching your systems? No, of course not. But what it means is that just because you are current on all of your patches it doesn’t mean you’re safe!

Here are some more commonalities in the attacks:

  • 98% of all data breached came from servers.
  • 85% of attacks “were not considered highly difficult.”
  • 61% were discovered by a third party(!)
  • 86% of victims had evidence of the breach in their log files(!!)
  • 96% of breaches were avoidable through simple or intermediate controls.
  • 79% of the victims that were subject to PCI/DSS regulations had not achieved compliance with the regulations. Admittedly, that means that 21% had achieved compliance, and were breached anyway, but why stack the deck against yourself? If you’re subject to the regulations, make sure you’re in compliance.

So what are the takeaways from all of this data? Although I would encourage you to download and read all 66 pages of the Verizon report, here are a few points to consider:

  • 86% of victims had evidence of the breach in their log files, yet 61% of the breaches were discovered by a third party. That suggests that, just maybe, we should be paying more attention to our log files. Now, I understand that there aren’t many cures for insomnia that are better than trying to parse through several servers worth of log files looking for anomalies. But that’s why there are automated tools these days that will do that for you.
  • SQL injection has been around for over ten years, and still causes a large number of data breaches. Here’s a high-level example: you have a form on your Web site that is intended to capture user input and stuff it into a SQL database. Maybe it’s the billing information for your on-line shopping cart. But instead of entering the data you’re expecting, an attacker enters a SQL language statement that’s intended to either extract data from the database, modify data in the database, or deliver malware to the system.

    You can’t fix this by applying a patch, modifying a setting, or changing a Web page. It’s almost always an input validation failure. That means you have to fix the code behind the application so that it actually validates that the information that’s being typed into a field is really the kind of information that’s expected. It isn’t necessarily easy, and it isn’t necessarily inexpensive. But data loss isn’t cheap, either.

  • The use of stolen credentials was the top hacking method used. Two-factor authentication (e.g., RSA’s SecurID), which can largely render stolen credentials useless, has been around for years. Apparently not enough organizations are using it.
  • One of the more interesting (to me, anyway) recommendations in the Verizon report is to filter outbound traffic. That way, even if malware does get in the door, you have some measure of control over what information leaves your network. This is sometimes referred to as “Data Loss Prevention,” or “Content Security.” Here’s what they had to say about it:

    Most organizations at least make a reasonable effort to filter incoming traffic from the Internet. This probably stems from a (correct) view that there’s a lot out there that we don’t want in here. What many organizations forget is that there is a lot in here that we don’t want out there. Thus, egress filtering doesn’t receive nearly the attention of its alter ego. Our investigations suggest that perhaps it should. At some point during the sequence of events in many breaches, something (data, communications, connections) goes out that, if prevented, could break the chain and stop the breach. By monitoring,understanding, and controlling outbound traffic, an organization will greatly increase its chances of mitigating malicious activity.

    By a happy coincidence, one of our primary vendor partners, WatchGuard, recently introduced a line of appliances that are specifically designed for precisely this task. I’ll be writing more about that in a future post.

  • Don’t assume that you’re too small to interest the criminals. 9% of the breaches were in companies with ten or fewer employees. Another 18% in companies with 11 to 100 employees. 23% in companies with 101 to 1,000 employees.

And, finally, don’t assume that the situation is hopeless. Remember that only 4% of breaches were judged to have required difficult and expensive measures to avoid. To quote from the conclusions of the Verizon report, “Configuration changes and altering existing practices fix the problem(s) much more often than major redeployments and new purchases.” We do have the tools to get the job done. We just have to make up our minds to do it.