You are here: Home / Archives for data confessional
post

Day 6 – Bubble Charts Delta

May 22, 2012 By Leave a Comment

Bubble Chart Showing Rack Top - Rack Bottom

In the bubble chart yesterday, we looked at rack top temperatures over the course of a week.  I mentioned that with bubble charts, it’s not easy to compare two parameters such as rack top and rack bottom.  However, what you can do is look at the delta temperature (Rack Top Temp – Rack Bottom Temp) for each location.  The bubble chart above is showing this difference.

The story that this data is telling us is incredible and extremely useful in determining airflow inefficiencies.

Here is some background. This facility has an under floor air distribution system with cold aisle/hot aisle layout.  Cool air is delivered through perforated tiles to the rack inlet.  Generally, the difference in air temperature between the bottom and top of the rack should be around 8°F due to the heating as the air rises and the typical mixing that occurs.

Takeaways

  • Rack 6, 8, 9, and 29 have greater than a 10°F delta T.  This may indicate areas where lots of mixing is occurring.  Most likely air from the hot aisle is sneaking over the top or through the racks affecting temperatures.  Blanking panels may help.
  • There are many racks with a negative delta T, meaning that the top of the rack is colder than the bottom of the rack.  Potential causes are  hot air from the hot aisle  infiltrating the cold aisle low to the ground due to missing blanking panels, through holes in the floor, or through the IT equipment itself.  The negative delta T may also indicate that IT equipment is installed backwards exhausting hot air into the cold aisles.
  • Most of the racks have a delta T that changes over the course of the week as indicated by multiple bubbles on a single vertical line.  However racks 12, 16, 17, and 18 show only one large bubble –  meaning that they are operating at exactly the same delta T for the entire time.  How could this be ?

Issues

  • The real value of looking at delta T in a bubble graph is to identify airflow issues..  This graph would most likely be used when hot or cold spots are identified to determine the root cause.

Previous Posts
About the Data
Day 1 – From 10,000 Feet
Day 2 – Rack by Rack
Day 3 – Learn to Love Graphs
Day 4 – Wordle
Day 5 – Bubble Charts
Today – Bubble Charts Delta
Tomorrow – A Day of Rest

Filed Under: Data Centers Tagged With: , ,
post

Day 5 – Bubble Charts

May 21, 2012 By 1 Comment

Rack Top Temp Bubble Chart

Bubble charts visualization takes accuracy to the next level because they can show three variables at one time.  In this case the parameters are rack, top temperature, and time. Each vertical line is a specific rack.  The graph above is displaying the top temperatures.  The bubbles on that vertical line represent the temperature of the rack during the time period (the temperatures are rounded to the closest 2°F increment).  The size of the bubble indicates how long the rack was at that temperature.  For example: the top temperatures at rack 6 were roughly 68°F, 70°F, 72°F, 74°F, and 76°F.  The larger bubbles at 72°F and 74°F indicates that rack is operating at those temperatures the majority of the time.

Takeaways

  • It is instantly obvious that racks 12, 17, and 20 have persistent hot spots.
  • There appears to be multiple racks that share the same temperature profile.  Racks 1 – 4 are all operating in the same temperature range.  Racks 5 – 10 are also similar.
  • Rack 15 seems to be an outlier operating cooler than its surrounding sensors.  Possibly an airflow issue?
  • Racks 31 – 34 are the coldest of the bunch.  There may be an excessive amount of floor tiles in this region.

Issues

  • Bubble charts can only show one measured parameter at a time. In this case, it represents rack top temperatures .Bubble charts are not good for comparing rack top and rack bottom temperatures
  • Also, bubble charts should be reviewed over a specific time duration.   The chart above displays data over the course of a week.  You could certainly look at data over the course of a month or longer, but it becomes more  difficult if you would like to compare two time periods such as one week versus another week.

Previous Posts
About the Data
Day 1 – From 10,000 Feet
Day 2 – Rack by Rack
Day 3 – Learn to Love Graphs
Day 4 – Wordle
Today – Bubble Charts
Tomorrow – Bubble Charts Delta

Filed Under: Data Centers Tagged With: , , , ,
post

Day 3 – Learn to Love Graphs

May 17, 2012 By Leave a Comment

Top of Rack Temps

Bottom of Rack Temps

Humans are visual creatures. Our ability to identify patterns is a strength that should be used when analyzing data…which means –  graph it!   These simple line graphs showing all 149 rack top and rack bottom temperatures sum up all the data in a comprehensive manner.  These visuals allow us to see every piece of data for the entire time period.  Note that I left off the legend in these graphs to save space.

Takeaways

  • The hottest and coldest racks in the data center can quickly be identified.  Looking at the top temperature graph, it is interesting to note that the coldest rack is significantly cooler than the rest.  I wonder why?
  • Beginning on 12/26 a large portion of the temperatures drop for about a day then spike, then return to their previous position.  Both the top and bottom graphs reflect this trend.  Some racks do not follow this pattern. Perhaps they are in a separate room?
  • The rack bottom temperatures have a much wider temperature disparity, ranging from 50°F – 89°F.  The rack top temperatures range from 58°F – 87°F.  In order to get 89°F at the rack bottom, I suspect there is some mixing going on in that location
  • The graph makes it quick and easy to identify that certain racks are operating above the upper thermal limit of 80.6°F.

Issues

  • Visuals provide a much friendlier way of looking at the data, but they can still be overwhelming.  If the operator wants to quickly identify which are the hottest, and thus most critical locations, he has to inspect the graph and do some mental thinking.  Tomorrow’s visual will leapfrog this step  altogether.

Previous Posts
About the Data
Day 1 – From 10,000 Feet
Day 2 – Rack by Rack
Today – Learn to Love Graphs
Tomorrow – Wordle

Filed Under: Data Centers Tagged With: , , ,
post

Day 2 – Rack by Rack

May 16, 2012 By 1 Comment

Rack by Rack Temperatures (truncated at 30 sensors for space)

This view shows the min, max, and average values for the top and bottom of each rack.  The actual table has 149 rows, but I cropped it for space purposes.  You’ll quickly realize that while this more detailed information is much more useful than yesterday’s table, it is difficult to extract any actionable information if we can’t sort this data.  Here is the same data table sorted to show the 20 hottest rack and 20 coldest based on their average.

Hottest Racks

Top Coldest

Takeaways

  • 13 racks average above the upper thermal limit of 80.6°F and 1 rack averages below the lower limit of 64.4°F.  The hot and cold spots have been identified.
  • On Day 1 we saw that the maximum rack top and rack bottom temperature were both 88.7°F.  Our question was if they came from the same location.  We now see that they don’t – rack 115 has a max top temp of 88.7°F and rack 20 has a max bottom temp of 88.7°F.

Issues

  • While this information is valuable, presenting it in a table is still overwhelming and difficult to quickly extract the most useful pieces of information.
  • Tables are useful for presenting static information but not very good at showing time dependent patterns.  Min, max, and average provide a good summary for the entire time period, however they cannot reveal any time dependencies.

Previous Posts
About the Data
Day 1 – From 10,000 Feet
Today – Rack by Rack
Tomorrow – Learn to Love Graphs

Filed Under: Data Centers Tagged With: , ,
post

Day 1 – From 10,000 Feet

May 15, 2012 By 3 Comments

Data Summary

Let’s begin by taking a very high level view.  The table above calculates the minimum, maximum, and average temperature for the entire data set and display it in a simple table.

Takeaways

  • Maximum temperature exceeds the ASHRAE recommended upper thermal limit of 80.6°F and the minimum temperature exceeds the lower limit of 64.4°F.  It is evident that there are hot spots…and cold spots
  • Average temperatures for rack tops and bottoms are within ASHRAE recommended ranges
  • Top temperatures are hotter than bottom temperatures, as expected, for min value and average value, but not for max value.  How come?

Issues

  • It doesn’t make sense that the top and bottom temperatures would be the same for the maximum value.  Are they coming from the same location?
  • You can’t determine which specific racks are outside of thermal threshold so there is no way of locating the hot spots. Although this information is at a glance simple, useful and high level, a data center operator needs more granular visibility into the statistics for each rack

Are there any other important takeaways you can identify in the data?  Leave your thoughts in the comment section.

Posts in Series
Yesterday – About the Data
Today – From 10,000 Feet
Tomorrow – Rack by Rack

Filed Under: Data Centers Tagged With: , , , , ,
post

Data Confessional

May 14, 2012 By 9 Comments

“If you torture the data long enough, it will confess.”

                                                   -Ronald Coase

I love that quote because it gives data a humanistic capability…the capability to obtain useful information. Inside every set of data, no matter how simple or mundane, there is information to be obtained. Data not only contains information that is obvious, but also information that we don’t yet know exists. When looking through raw numbers, charts, and graphs from every perceivable angle and combination, there comes a moment that these small but important pieces of information will emerge. With persistence, an open mind, and a will to engage in a bit of targeted torture, you can make a little bit of data go a long way. Over the next seven days, I will take a set of seemingly simple temperature data and massage it, analyze it, and torture it until it confesses what it knows. Each day I will present the data using a new visual. I will show you how, by analyzing and displaying data in different ways, you gain entirely new actionable information and surprising insights.

About The Data

The data that I will be analyzing is one week of rack inlet air temperatures obtained from an intelligent energy management system in a data center. Temperatures were measured in the cold aisle at the top and bottom of the rack. The data was collected with 149 sensors resulting in 298 monitored points in the data set (149 top temps and 149 bottom temps). The data is comprised of hourly averages for each point for the duration of the week. Below is a sample of the raw data.

Raw Data (Top of Rack Temps)

Raw Data (Bottom of Rack Temps)

Tomorrow will be the first day of analyzing this information in the attempt to gain insightful and actionable information.  Stay tuned.

Tomorrows Post: From 10,000 Feet

 

Filed Under: Data Centers Tagged With: , , , , , , , ,