50Micron.com

Ok, that was too cutsey for such a classy establishment.

When you’re building a SAN, everything should play together, in the same SAN box if you will (with my apologies to QLogic.)

When you start putting in multiple stand-alone SAN islands you increase your maintenance overhead exponentially.  You also prevent the very thing that make a SAN a huge advantage over DAS.

Everything can see everything.

If you have a host and need to throw a certain type of storage at it, you can do that easily.  (and if it’s already cabled you can do it from your living-room)

However, if SAN A/B are connected to one group of hosts/storage, and SAN C/D are connected to a second group of storage, and SAN E (no redundancy) is connected to even a third, you run into a problem.

*NOW* if I want Host_A (Connected to SAN A/B) to see Storage_C (Connected to SAN C/D) I have to do much more than a simple zoning change.

In the end, this is where a few well placed ISL connections can come in handy.  VSAN them off so they don’t cause the fabrics to merge, create an IVR zone to route across them, and then presto.  Host_A can see Storage_C with a minimum of fuss.

Or maybe a *REAL* core-edge topology even.  Where you put a core switch with 24-port (fully subscribed) blades ISL’d to an edge switch, which maybe has the 4/4/40 configuration (4 8-Gig ports, 4 dedicated 4Gig ports, and 40 shared 4Gig ports)

And put one person in charge of it.  Preferably someone with a touch of OCD.

I was whiteboarding a switch migration today with one of the DCR people, and it occurred to me:

If you can visualize the data’s path through the system, your life gets a lot easier. 

If you can see the path the data is going to take through the San, I.E. from Switch1, Blade1, Port4 through an ISL to Switch2, to Switch 2, blade 2, port 16.  you put yourself in a better position to shorten that path.

First, cut the ISL’s.  Unless you can absolutly avoid it, don’t pass data down an ISL link.  Pushing data through an ISL does two things.  It creates a bottleneck where there may not be enough bandwidth to handle multiple hosts.

Second, keep the intra-switch hops to a minimum.  If you can do it, plug the storage and the higest performing hosts into the same grouping of ports.  Most switches use 4 port ASIC’s, and if you look at the switch you’ll see that the ports are grouped by ASIC.

However, the 32port Cisco blade is an exception, uses basically the same ASIC, but shares the bandwidth over 8 ports, so keeping that in mind it’s best to connect the lower performing hosts (Windows?) to those ports, and definately *DO NOT* connect ISL’s to the 32 port blades.

Well, I may stand corrected yet again.  After countless attempts to go back and forth with EMC about “Dial-Home” on the Cisco switches, someone has finally produced a document that seems to show how to set up Dial (or in this case email) home on the Cisco switches.

I’ve got the “Cisco MDS900 Cookbook for SanOS v2.x”  Which seems to spell out in blinding detail how to configure email-home.

The only shortcoming is that this is a Cisco document and EMC failed, again, to give me the information relevant to getting it to email EMC support.

What my research seems to be telling me is this.

Until we upgrade to v3.x SanOS, we’re out of luck, though limited email home can be gotten THROUGH EMC Control Center, but the EMC SAC is apparently not really set up to take emails from ECC, as there isn’t enough relevant information for them to act on.

So we’re back to “have ECC page you, then you call the SAC when there is a problem.”

I’m going to try and get my 9216′s upgraded to 3.0(2) before we get too much in the way of “production” on them.  Maybe then I’ll be able to configure it properly, instead of the kludge they’re trying to foist off on me.

 And just in case you need it, here are the email-home instructions for SanOS 3.x.

http://www.sangod.com/wp-content/uploads/2007/01/emcemailhome.pdf

Cisco has released version 3.0(2) of their firmware, and with this, comes, finally, an EMAIL-HOME feature for the switch.  It’s not perfect (because unlike the dial-home on the Symm, this is dependent on an external server or two, and doesn’t work if the network connection is down.

The long and the short of it is the Cisco Fabric Manager, which is installed on whatever host you’re going to manage the switch from (as it is with the current versions of the cisco SanOS) monitors the switch via SNMP and when an error condition is detected, sends an email to EMC with the appropriate information.

I want to say it’s about bloody time.  Cisco switches have been the red-headed stepchild of EMC for too long.  it’s about time they supported them with the fervor that they do the rest of their products.

The down-side?  You have to go from version 2.1(x) to 3.0(x) and I’ve not heard back from EMC at this time as to whether this is an online upgrade in anything shy of the MDS95xx switches.  (I’m using MDS9216′s, and an MDS9140, which I suspect need a full reboot to come up on the new firmware rev)

 

Ok, there is something I hate about Cisco- and it’s not really about Cisco, it’s about EMC’s complete failure to completely support their MDS series of switches.

First – When you buy an MDS 9xxx switch from Cisco, it comes with the ability to dial into Cisco when there is a problem, much like EMC does with the Symmetrix.  This feature is not available when you buy the switch from EMC.  (IE a Cisco switch with a $10,000 sticker on it that says “EMC”) you don’t get this ability.  In fact, you get no dial-home capability at all. 

I called in to the SAC, talked to our sales team, our former Project Manager who was (sort-of) handling the implentation, etc, and the long and the short of it is that in order to be notified when there is a problem with the switch, you have to set Control Center up to email you, and then you have to call EMC.  (I’ve called them so many times I know the number by heart)

Secondly, (and more to the point of the subject of this email) making a zoning change in ECC requires that you log into the switch and type “copy run start” at the prompt to save the configuration to the start-up.  I’ve known people to go months without doing this – only to find that every zoning change made in the last number of weeks is lost on the reboot of the switch.

And Third – when you do call EMC for support, they have to wait on Cisco to ship the parts.  So technically you’re not even really getting EMC support, you’re getting third party “ghost” support.

This seems to me to be inexcusable considering the number of MDS switches that EMC is putting on the market.

My advice – if you’re going to buy a Cisco fibrechannel switch, and I do recomend them, buy it from Cisco.  Until EMC decides to support them fully, I don’t think it’s worth the convienence of getting everything from one place given what you come up short on.