Central Datacenter Cleanup – Before And After

A couple weeks ago I flew back to Oklahoma to work on some critical network changes.  Over the last few years, our Central Datacenter has gone from a 2-post rack and folding table to about 6 racks full of gear.  As it has grown, the cabling has become a nightmare.  Here are a few "before" shots taken by some Church IT friends about a year ago.

2390494756_50001bd46d

The picture above shows our EMC SAN (left) and the spaghetti bowl of cables (right) and all sorts of single/individual servers (middle).  All of this has been virtualized now on VMWare ESX 3.5.

2389662297_0b9e722782

This picture above shows the crazy cable mess coming through the ceiling and winding across multiple racks and "ladders" connecting the racks to the wall.  What a mess.

2383949416_4e4634eb9f

Here's a better shot, above, of the spaghetti cluster of cables.  Under all those cables are about a dozen Cisco devices – routers, switches, firewalls, and the "core" 4506 that was replaced on this trip.

Basically, here's what happened during this trip:

  • Bossman Brad and co-worker Evan moved all the punch/panels from the spaghetti rack and created a nicely dressed wallmounted setup.
  • The 4506 was decommissioned
  • The 6509 was put in place
  • All the necessary Cat5 cables were re-routed from the wallmount to the 6509.
  • All the necessary Fibre cables were re-homed into a wallmount box, and then cleanly re-routed to the 6509.
  • Our AT&T "gigaman" (Gigabit Metro Ethernet) was brought up
  • The 4506 was moved to our OKC Campus – and gutted – and turned into a 4507

Here are some "after" shots of what was accomplished.

4483870

The rack on the far left is our "DR" rack – it has a 108TB Nexenta SAN (Using Pogolinux/Supermicro Z400 boxes) in it which we'll use to replicate our XSAN and EMC SAN.  It also contains our "DR" ESX cluster and any routers/firewalls/devices necessary to be completely ready in case of a disaster.  The far right shows the wallmounted Cat5 cables.  The two racks on the right house the primary ESX Cluster (after virutalization) and the 6509 and any core Cisco gear needed at the Central Datacenter.

4484316

This picture above shows our nicely dressed wallmount Cat5 cabling and fibre cabling.  Evan and Brad did an awesome job here.

4484047

Above is our new-to-us Cisco 6509.  Blue cables are normal cat5 cables going to clients or phones.  Green cables go to servers.  Purple cables are iSCSI.  Orange is fibre.  We were able to consolidate most of our Cisco equipment to the 6509, ASA 5520 (firewall), 2811 (PRI/voice), 3725 (SIP/voice) and 3825 (public facing router).

4489587

And, finally, here's the 4506 – now re-configured and re-homed in a 4507 cabinet at our OKC Campus.  In this picture it's only connected to the AT&T Gigaman (the equipment in the white AT&T box) via the orange fibre cable.  In the next few days, it'll be brought fully online in time for the Dave Ramsey "Town Hall for Hope" event happening at the OKC Campus.

There's more to come – some Internet changes, internal routing changes, and external BGP setup too.  More to come in the next 90 days.

4 thoughts on “Central Datacenter Cleanup – Before And After

  1. Hii, hw r u..?

    i just want to contact a person who is working on this datacenter, please i need some help, will u provide me your any contact like email address..

    thank you

  2. Hi,

    I notice you cabled the 6509 down the middle. Are you concerned that it could be an issue if the fans failed an you needed to replace the tray on the left? Just wondering because I am looking into what the best way to cable a 6509.

    thanks

  3. BM–

    I have to be honest, I never even gave the fan tray on the left a moments thought. Now I’m paranoid 🙂 Thanks for giving me added stress! 🙂

    –DW

Comments are closed.