Originally Published: Wednesday, 10 January 2001 Author: Alexander Reelsen
Published to: enhance_articles_sysadmin/Sysadmin Page: 1/1 - [Std View]

RRDtool - An efficient way to store and display data

Looking for a simple way to chart network activity and system status? Alexander may have the answer you're looking for with this article on RRDtool.

You have become the network administrator of the company where you work . You sit relaxed in your chair, when the boss enters the room. He begins to cry that he doesn't understand the thousand-word e-mails he receives daily concerning IP traffic statistics. He would rather see wonderful graphical statistics, which exactly reveal the same as your mail, but in a more appropriate way (for your boss).

Now you have either the choice of writing everything from scratch yourself or you choose an already existing tool. A very good choice would be RRDtool (RRD stands for round robin database). Many of you do know MRTG, the multi router traffic grapher, a tool, which reads the amount of traffic that occurred on an interface of a router, writes this data into a file and creates an image of it,showing the data flow in this interface for a certain timespan.

RRDtool, also written by Tobias Oetiker as MRTG was, is basically a new implementation of the logging and graphing features of MRTG. RRDtool itself is not capable of collecting any data. RRDtool can save data in time-series. That means you define a RRD database and feed it afterwards with time-value pairs.

The technical background of RRDtool is a somewhat more complex. A database created with RRDtool consists of several datasources (referred as DS's in the future) and Round Robin Archives (referred as RRA's from now on). Furthermore you have to supply the start time of the database (beware: RRDtool cannot handle times before 1970) and the interval with which data is going to be inserted into the RRD. A DS is a definition for data (it can be incoming traffic on the wire), while the RRA is used to store the real data. I will not explain the way data is stored in the RRD, as this is only an introduction to RRDtool. If you want to know this, please consult the RRDtool manual.

Now that we know what we need, let's create something nice. First, we need to obtain RRDtool from http://www.ee.ethz.ch/~oetiker/webtools/rrdtool. After downloading, unpacking and installing it (it is quite easy because it uses autoconf; you normally should have no troubles with ./configure ; make ; make install, if your distribution does not ship RRDtool).

Creating and feeding a rrdtool database is the next important step, after which we can create nice graphical output. Imagine the following are data values fetched via SNMP in a 5-minute interval:

|   Time    |Incoming packets| Outgoing packets |
|           |                |                  |
| 978303900 |      500       |       100        |
| 978304200 |      600       |       400        |
| 978304500 |      650       |       700        |
| 978304800 |      660       |      1000        |
| 978305100 |      800       |      1400        |
| 978305400 |     1000       |      1400        |
| 978305700 |     1100       |      1400        |
| 978306000 |     1300       |      1400        |
| 978306300 |     1500       |      1600        |
| 978306600 |     1800       |      1800        |
| 978306900 |     2100       |      1900        |
| 978307200 |     2100       |      2200        |
| 978307500 |     2300       |      2400        |
| 978307800 |     2600       |      2600        |
| 978308100 |     2900       |      2600        |
| 978308400 |     3100       |      2700        |
| 978308700 |     3200       |      2700        |
| 978309000 |     3400       |      2800        |
| 978309300 |     3800       |      3200        |
| 978309600 |     4000       |      3300        |
|           |                |                  |

Now this is the data, we will put into the RRD. The rrd file itself has to be created with this command:

rrdtool create packets.rrd -b 978303600 -s 300 DS:incoming:GAUGE:600:0:U DS:outgoing:GAUGE:600:0:U RRA:AVERAGE:0.5:1:100<

This command creates the rrd file packets.rrd beginning at the -b parameter (newyear 2001 has been chosen) and has 300 second lasting steps. Two datasources for incoming and outgoing data are specified. GAUGE is the datasource type and is simple to count (please read the manpage for other types of datasources). 600 is the heartbeat, which defines the number of seconds between updates before this value is set to U (means undefined, what is not equal 0). The last two values are the minimum and the maximum values, data can have. One RRA has been defined, which can contain exactly 100 values. If you enter value 101, value 1 will be overriden, be aware of this and create your archives big enough. You can create further RRA's which sum up other RRAs to save records (if you put 2 values into 1 average). Now I'm going to put the table data in to the RRD (imagine I would have insertedall the data this way):

rrdtool update packets.rrd 978303900:500:100

But now is time for some graphing:

rrdtool graph packets.png -s 978303900 -e 978309600 \
--title "Traffic analysis for new years eve 2001" \
DEF:incoming_packets=packets.rrd:incoming:AVERAGE \
DEF:outgoing_packets=packets.rrd:outgoing:AVERAGE \
"LINE3:outgoing_packets#AA0000:Outgoing packets"
"LINE3:incoming_packets#00FF00:Incoming packets"

Now let's play a bit a bit with the graphing functionality of RRDtool. You can stack both values, so one sits on the top of the other. This is useful to see the complete amount of traffic. In addition I exchanged the lines with areas.

rrdtool graph rrd-example2.png -s 978303900 -e 978309600 \
--title "Traffic analysis for new years eve 2001" \
DEF:incoming_packets=packets.rrd:incoming:AVERAGE \
DEF:outgoing_packets=packets.rrd:outgoing:AVERAGE \
"AREA:outgoing_packets#AA0000:Outgoing packets" \
"STACK:incoming_packets#00FF00:Incoming packets"

The next part is one of the trickiest functions in rrdtool. You can apply mathematical operations to the data before it is drawed. In the following example I will simply multiply the data by -1, so it will be negative and you can see the x-axis as the symmetric axis between incoming and outgoing traffic. And I added a vertical rule, which marks what is written in the legend.

rrdtool graph rrd-example3.png -s 978303900 -e 978309600 \
  --title "Traffic analysis for new years eve 2001" \
  DEF:incoming_packets=packets.rrd:incoming:AVERAGE \
  DEF:outgoing_packets=packets.rrd:outgoing:AVERAGE \
  CDEF:incoming_packets_new=incoming_packets,-1,* \
  "VRULE:978309000#0000FF:Half past one after new year 2001" \
  "AREA:outgoing_packets#AA0000:Outgoing packets" \
  "AREA:incoming_packets_new#00FF00:Incoming packets"

What I showed you above is only a very limited introduction of RRDtool.It has far more abilities, what makes it really worth a look. There exist frontends which do all the hard work, RRDtool is not able to do,like fetching the data from routers via SNMP or executing commandsand bring the output into the right format for inserting it into the RRD.Below there is a list of some useful links for people who would like to use RRDtool. Take a look at the RRDtool homepage and you will find a gallery with cool images created with rrdtool in different situations.

URL List