50Micron.com

Ok – certain people do certain things well.

I’m a storage administrator/architect.  If you present me a problem I will *ALWAYS* look at it from a storage standpoint.  If you present me with a non-storage problem, I’ll try and make it fit.

I’ve identified four types of systems engineer-type-people:

Storage people

Server people

Network people

Desktop people

I think that just about anyone in IT either fits into one of those four roles or supports one of those roles.

Now when you are looking to solve a problem, the solution you get depends on who you go to.  If you ask a desktop person to solve a network problem for instance, they will probably come up with something under the desk.  (IE throwing a linksys router under a desk.)

If you try and throw a server person a storage role, you’re going to get a server solution to that role.

Enter IBM GPFS.

GPFS is a server solution to a storage problem.  It’s obvious that the person who came up with the idea of solving a storage problem by loading software on a server is not a storage person.

POSIT:  Mutliple hosts in a web-farm need access to data.  Filesystems need to be R/W to an ingest server and R/O to the web-content servers.

Storage Solution:  NAS/NFS – Trunked connection to a real backbone and multiple Apache webserver front-ends running at 1G to play out data.  (Fastest data transfer is going to be the 45MB/Sec backbone coming into the building, so a single Gigabit connection can handle it.   F5 Round-robin load-balancer to distribute the front-end load.  (might also be proposed by Savvy network people, who tend to understand NAS)

Server Solution:  IBM GPFS solution.  Over a million dollars in net-new server hardware + software licensing (not including storage).   Each host accessing storage requires HBA’s, Drivers, fast RELIABLE network. and a level of complexity unheard of even in government.

From what I can tell, and maybe someone can give me a little more insight, works very much like Sun’s Shared QFS.  A metadata server acts as a gatekeeper telling which member servers can access which blocks on which disks.  There is still no simultaneous disk access because a SCSI lock is a SCSI lock.

Now from a storage standpoint, this is rife with problems.

First off, it would seem that if network access was compromised during a write data integrity could easily be compromised.

Secondly, Other than block-level mirroring of the underlying disks, I can’t see a good way to replicate this.  And block-level mirroring of the underlying disks would require an identical infrastructure at the remote/DR site wouldn’t it?  That is of course assuming that the metadata can be mirrored.

Now in database uses or other types of distributed computing I can see it being VERY valuable.  But for flat file storage and web retrieval I can’t think of a single good reason to use something so obnoxiously complicated.  Especially when EMC Celerra, NetApp, or just about any of the other higher-end NAS appliances would cost *SO MUCH* less and be *SO MUCH* more reliable.

/EndOfRant

Ok, given the changes to the storage arena I’ve been working on a revised “Tiering system” to incorporate all of the levels of data…importance?

My version of Storage Tiering is (or should be) as follows:

• Tier-1    – Symmetrix/Replicated – High Performance/Criticial Data
• Tier-2    – Symmetrix/NonReplicated – High Performance/Non-Criticial Data
• Tier-3   – Symmetrix/SATA/Replicated – High-Medium Performance/Critical Data
• Tier-4   – Symmetrix/SATA/NonReplicated – High-Medium Performance/Non-Critical Data
• Tier-5    – Clariion/FC/Replicated – Medium Performance/Critical Data
• Tier-6    – Clariion/FC/NonReplicated – Medium Performance/Non-Critical Data
• Tier-7    – Clariion/SATA/Replicated – Low Performance/Critical Data
• Tier-8    – Clariion/SATA/NonReplicated – Low Performance/Non-Critical Data
• Tier-9    – CelerraNAS/Replicated – Network Attached/Critical Data
• Tier-10  – CelerraNAS/NonReplicated – Network Attached/Non-Criticial Data
• Tier-11  – Atmos – Network Attached / Low Performance
• Tier-12  – Centerra (Content Addressable Storage) – Low Performance Archive / Highly Available
• Tier-13  – Primary Tape-In-Library (Automatic loading on demand via HSM)
• Tier-14  – Primary Tape-Out-Of-Library (Manual Intervention Required)

“Critical Data” vs. “Non-Critical Data” is simply a matter of how long you can be without the data should a failure or accidental deletion occur.  As all data is available in Tier8/9 storage (in theory).

I’ve also considered using Tier1/Tier1B to describe DMX storage vs. Clariion storage, given that there is a LOT of overlap in performance characteristics these days…

Oh, and iSCSI would be somewhere between 10 and 13….

Any thoughts?

I just got the following job posting and it made me, literally, laugh out loud, spitting latte all over my laptop.

If your sales rep allows you to do something like this, it’s a fair bet that s/he hates you (or is planning to buy your company out of bankruptcy later).

“WANTED: VMWare 1-month resident to assist with new deployment/planning around 200VM’s and new Celerra NS480′s being purchased by client. Will probably end up primarily being VM’s using NFS on NS Celerra Replication will be enabled between (2) NS480′s.”

The key points are:

200VM’s

Celerra

**NFS**

Replicator

Ewww…..

Did I mention NFS?

Someone actually sold this?  Even if the customer comes to you direct and says “this is what I want…” the answer should be “In the interests of protecting you from yourself, I can’t allow you to do this.”

I don’t care how much the deal is worth.

Got my first presentation on EMC’s new “Atmos” storage platform.

Now granted this was kind of a sales-ey (is to a word) presentation but I’m pretty impressed so far.

It seems what EMC has done is combined the best of Celerra and Centerra. (In fact, the gentleman giving the presentation sort of placed it on the map right between the two)

The basics of it is they get a bunch of 1U (Presumably Dell) Pizza-Box type servers and put them in front of a bunch of really *REALLY* cheap storage.

They then present the storage out using a variety of protocols, CIFS/NFS, and the REST/SOAP API’s.  Rumors of an iSCSI could not be confirmed…or explained (how in the world would you convert block-storage to object-storage and expect any kind of real performance?)

Downsides….well, there are multiple single-points-of-failure in each frame, which is why when you invest in the Atmos hardware you will buy a minimum of two frames.  I think this could have been avoided in a more robust deployment.

There is no “Compliance” edition (yet?)  This would/could easily be the replacement for the Centerra, if they can just get past that little hurdle.  I’ve known many customers (and been one myself) who have chosen the NetApp filer over Centerra for archiving because all we wanted/needed was a CIFS share that we could guarantee the content on.

I was not able to get reasonable performance numbers from the presenter.  Assuming Gigabit-Ethernet off the internal switch/bus/apparatus maximum sustained transfer rate would be 125 MBytes/Sec.  10Gig-Ethernet is currently running at substantially less than the 1.25G that you would expect.

I’m curious as to what the world’s thoughts are on “Cloud” storage (I hate the term “Cloud” anything – it’s a mostly meaningless term that describes nothing but outsourcing.)

Next step: Get my hands on one and try it out.  This may not be as much of a long-shot as it seems. 

There comes a time in every geek’s life….

A look of wonder crossing his face….

A new toy, a new project, a new endeavour….

A beloved wife saying something along the lines of “You bring that damned thing in here and I swear to god…”

This is one of those times.

Over the course of the next few weeks I’ll be moving 50micron.com and it’s assorted supporting systems off of the “OLD” storage (the 8 year old Dell PowerVaults I’ve been running on) and moving to a brand new (for me) NS500..

Now a few things I plan on making happen.  This NS500 will become an NS500G and CX500, because there is no way in hell I’m going to continue to live with a captured back-end when there is so much fibrechannel storage to be had, just out of reach..  (No iSCSI in this household, period, end of discussion)

Almost 9TB of storage, between the rack of 73G drives (pictured, sitting on top of the NAS) and the rack of 500G SATA disks (not pictured, too heavy for the bakers rack it’s all sitting on right now.)

Ohboyohboyohboyohboy.  This is the kind of thing real geeks live for.

Well – after moving along for a few weeks at a nice leisurely pace, I find myself working on six different projects.  Loads of fun, especially when three of them are just similar enough to get the details confused.

Got an iSCSI install next weekend though, this should be interesting.  I think I have a handle on how the Celerra does iSCSI, so the only real trick will be setting up the hosts correctly.  It’s a mixture of RHEL3, RHEL4, and CentOS5, which makes it even more interesting.

Another thing I got to play with last week was the McData Eclipse series FCIP router.   When tied to a pair of Brocade switches (one on each side of the FCIP tunnel) I found them to be almost impossible to use.  I’ve got quite a bit of FCIP experience in different replication scenarios, and it still took me almost 6 hours to get these connected.  Talk about having to pay attention to detail, this was painful.  I’d be infinitely happier with a Cisco 9216i with a 14/2 blade in it and be done with it.  Eliminates the need for 90% of the make-work that had to be done to get this thing running.

In McData’s defense though, I came in unprepared, it was my understanding going into the engagement that the McData was already set up and I came in and found not only wasn’t it set up for FCIP, (it was set up for iSCSI) the tunnel hadn’t been built.  So all we really had was a CE who came in, set the IP’s and ran like heck for the door.

I also had a Celerra NSX install that same weekend.  The NSX is an interesting piece, very much like the old CNS boxes, albeit much smaller/faster.  Modular setup makes it very expandable.

What I don’t get, is why, in this day and age, you are still required to use a floppy to boot the control-station installation CD.  Bootable CD’s have been around for quite some time, and in fact the NS502g I ran when I was at the evil empire even booted perfectly off the CD.

The NSX required however a serial console connection into the server, and had the bios locked so you couldn’t change the boot order.  Add that to the fact that EMC put the wrong crossover serial cable in the box with it (Mail-Female) meant a 2 hour install ended up taking 8 hours when you factored in driving around looking for a USB floppy drive (to create the boot disk) and a null-modem adapter (First time I’ve set foot in a radio shack in 10 years).

This week I’m off to sunny (?) Florida to do an ECC install and then off to Texas to do the NS20/iSCSI install.  *THAT* should be worth writing about.

Thanks.