Wednesday, February 10, 2010

Citrix Printing Fun

So,  I suppose the most natural place to start writing my blog is to talk about the challenges I've most recently been dealing with.

Part of my job is supporting a Citrix environment for our hosted clients. These clients essentially host all their desktops and applications in our datacenter. It is really a great value for most of our customers as it allows their IT to be completely "hands off". Most of these environments are very simple 1 or 2 virtualized server environments that run quite well. One of our hosted clients started to develop a very nasty print spooler crashing problem, they were calling the helpdesk 10+ times a day because their printers were missing. This Citrix environment had historically been problematic in nearly every other way, and it was just a matter of times before these printer issues surfaced.

The previous Citrix admin for this organization set a very good example on how not to do things. I learned quite a bit from this ordeal, these printers are mostly running great now, but it took some doing.

Citrix Policies
The Citrix printing policies are the first place to start building how you want your Citrix printing environment to work. The policies that ended up working well in my environment were the following.

Client printer auto-creation - I set this to "auto-create all client printers" I initially thought I would like this to be set to auto create all client printers except for network printers, and in an environment that has scripted installation of network printers I would. But in this environment network printers are all manually mapped as needed (I know... it's not optimal) So, in this case I wanted to map my network printers through the ICA session.

Native printer driver auto-install - I ended up setting this setting to "Do not automatically install drivers" Essentially what this policy does is if you install a printer that Windows already has a native driver available for, it will go ahead and use it. We want to monitor as close as we possibly can which drivers are installed on our Citrix servers (and you do too!) so we do not allow native drivers to be automatically installed.

Universal Driver - I set this to "Use universal driver only if requested driver is unavailable". THIS DOES NOT MEAN that if your printer doesn't match a print driver on the server that it WILL match the universal driver. The universal driver works well for alot of printers. But it certainly does not work for everything.

There are a bunch of other Citrix Printing policies, but these are, in my opinion, the most important.

Print Drivers
Print drivers are like alot of other things, you want to "Keep it short and sweet" Install as few drivers as you can to do the job. A bad print driver will wreak havoc on your farm and it is alot easier to weed out a bad driver amongst 25 drivers than it is amongst 150 drivers.

I have compiled a list of drivers that tend to work really well in my environment. My base list came from this Brian Madden blog. But I have expanded upon it.

  • Canon Bubble-Jet BJC-210
  • Canon Bubble-Jet BJC-4000
  • Generic / Text Only
  • Epson Stylus COLOR ESC/P 2
  • HP Color LaserJet
  • HP Color LaserJet 4500
  • HP DeskJet
  • HP DeskJet 550C
  • HP DeskJet 850C
  • HP DeskJet 855C
  • HP LaserJet
  • HP LaserJet Series II
  • HP LaserJet 1100 (MS)
  • HP LaserJet 2100
  • HP LaserJet 4
  • HP LaserJet 5
  • HP LaserJet 6P
  • HP LaserJet 4000 Series PCL
  • HP LaserJet 8100 Series PCL
  • HP OfficeJet
  • Lexmark Optra
  • Lexmark 1020 Color Jetprinter
  • HP Laserjet 2035n
  • HP Laserjet 2015dn
  • HP Laserjet 6L
  • HP Laserjet 4M Plus
  • HP Universal Printing PCL5e
  • HP Universal Printing PCL6
  • HP Universal Printing PS

I know what you're thinking as you get to the end of the list. "HP Universal driver... really? ... that seems like a bad idea" I thought so too! But... then I found this fantastic HP document that outlines all the drivers they support on Citrix in different Operating Systems. It lists the Universal Printing drivers, and as far as I know, I haven't had issues with them yet.

It is important to note that although this document lists a few host based printers that HP considers compatible with a Xenapp environment, it is generally considered bad practice to implement a host based printer in either a Citrix or terminal services environment.

 
Remapping confused print drivers

So when I first realized I should be able to run my whole farm on a couple dozen drivers I was really excited, but of course skeptical. How could a Laserjet 4050n know it is supposed to use a Laserjet 6L driver? The good folks at Citrix worked that out for you a long time ago!

In the Citrix Advanced Configuration console if you right click on printers you will see the "Mapping..." button. This allows you to map the name of the driver your printer "thinks it wants" to the one you tell it it wants. But, this must be exactly what the Citrix client is looking for or your mapping won't work. Laserjet 4050n PCL6 is different from Laserjet 4050n PCL 6.

To find out what drivers your printer wants to use take a look in the Application log on your Citrix server. You will see an error 1007 logged for any printer that fails to map because it can't find a matching driver. The name of the event is "Metaframe Events" It will say Client Driver: drivername. That is the name you want to use in the advanced configuration console to remap to a driver that is installed on your server.

Printing Bandwidth!!
I rarely ever see this in any article when people are talking about Citrix printing problems. Citrix documents how important it is in pretty good detail, but I think alot of admins neglect to think about it, I know I did at first.

By default, a Citrix session will let you use as much bandwidth as you please to move that print job across the pipe to the printer. Have users wondering why their machine/app seems to lock up when they send a print job? This is probably why.

If you right click on your Citrix servers in the advanced configuration console and click properties you will see a tab to restrict printing bandwidth for each server. I would love if this was a farm level setting, or at least zone level since I think that makes more sense.... but its not. I started my servers out at 200kbps per server and that seemed to help out alot. I knocked it back to 60kbps recently and nobody has noticed. I think I will keep it there.

Keeping the print spooler and Citrix Print Manager services in check
So you're being a good Citrix admin and you've trimmed your print driver inventory down significantly, cut back your printing bandwidth, and even setup a few print driver mappings. Life is much better, but you still have what seems to be crashing spoolers...

You probably still have a funky driver. Its not uncommon to have a bad driver. But you don't really want to just start blowing away drivers.

In almost any troubleshooting situation I find it helpful to define the scope of the problem.

Is it one user? Do they have a certain printer?
Is it happening on all of my servers?
Is it the spooler or the citrix print manager service crashing?

None of these are very easy answers to find. I wrote a little script to help me out. All it does is write to a log file on a network share when a service stops and then restart the service.

echo "The print spooler service has crashed on" %computername% "at" %time% "on" %date% "....restarting the spooler" >> file://servername/sharename/spoolerlog.log
net start spooler

I then attached this script file to the spooler and citrix print manager service by configuring the services to "run a program" on failure.

Note- the name to start the citrix print manager service through net start is cpsvc. The Citrix Print Manager service does not automatically restart itself by default when it fails by default!

Hopefully this type of logging will allow you to cross reference other entries in your event logs to determine what is breaking your spooler/print manager.

In the event you find it is a driver that is working fine on another server you can use the advanced management console to replicate the driver from the working server over top the one on the broken server.

I wrote some other neat scripts that do stuff like manually remove print drivers and automatically install my list of native print drivers, I will post those up at some point.

Reworking Citrix printing an a poorly setup environment  can be a nightmare, so do it right the first time and it will be a breeze.