When an Exchange Server doesn’t Exchange…

“Well the high sheriff, told his deputy, won’t you go out and bring my Lazarus?”

Why am I quoting the Po Lazarus tune, the opening song of “O Brother Where Art Thou?” when I’m supposed to be talking about the Exchange Server that doesn’t?  It’s the Chewbacca defense!  This Exchange Server is so hosed I can comfortably quote an old folk song instead of talking about the server…

Ok ok, I’ll talk about the server:

This server is a Windows Server 2008 R2 SP1 Server running 10G E cards to talk to storage and it performs like it shouldn’t.

image

 

And Holy Moly!  DPCs consume more CPU than any one thread on the box.  Googly moogly!  We’ve got a problem here.  But why?  What does it mean?

Right Click this graph and select Summary Table:

image

Here we go, our DPCs are in SYSTEM (4), module elx_octeamvlan.sys.  But wait, there’s more, why?

image

Seems this driver in SYSTEM is spending a lot of DPC time on cores 6, 0 and 4.  Odd.  Lets see what else we can find to help them write a better driver:

DPCs are high, way too high:

image

Observe, DPC count is low on 6/4/0 cores, but waits are um, not low:

image

Huh, lets see what it is (symbols didn’t resolve sorry, but its NDIS.  The Driver / Hardware is a 10G E adapter:

image

Same function call in each of the three cores, lots of wait times.  We’re having trouble with the drivers implementation of how they talk on the network via NDIS.  They are aware and I believe have already fixed the problem.  Woot!  Another happy customer.

2 Comments

Leave a Reply