CAIDA CoralReef Exercises
Using the CoralReef C API
CoralReef is a CoralReef is a comprehensive software suite developed by CAIDA to collect and analyze data from passive Internet traffic monitors, in real time or from trace files. Full details are provided on the http://www.caida.org/tools/measurement/coralreef/. This exercise is one of a set designed to introduce you to CoralReef, providing a `hands-on' experience of analysing network data.
Before you can use CoralReef it must be installed on your Unix or Linux system. A copy of the CoralReef distriubution package is provided on the IEC CD, and these exercies have all been tested using it. Alternatively, the current version is available from the CoralReef web site.
Installing the software involves running its autoconfigure script,
running make depend and make to build the libraries and
applications, then
make install to put the CoralReef material
into appropriate directories. Again, full details of the install
process are given on the web site.
By default, CoralReef is installed in the
/usr/local/Coral
directory. To simplify running CoralReef application programs,
the /usr/local/Coral/bin directory should be included in your
PATH environment variable. Once this is done the applications
can be run by entering their name, followed by the name of the
trace file(s) they are to work on, e.g.
crl_info ODU-962010098.crl.enc
Medium.
To get the most from this exercise a student should have:
Obtain or locate the following traces:
A large number of ``packet header traces'', containing the first 48 bytes1 of each IP packet are available at http://moat.nlanr.net/Traces/
In addition there are some traces that have been selected specifically for these exercises. Your instructor may have made these available locally. They are also available at:
The traces normally include data from two interfaces, one collecting data from each direction.
The names of these traces consist of a three letter code, a Unix timestamp, and an extension indicating the format of the trace. The three letter code identifies the location of the monitor. For example ODU-947926964.crl.enc refers to the Old Dominion University vBNS link collected at 01:02 on Saturday January 15, 2000.
If you need to convert the Unix timestamp to a date and time try:
perl -e 'print scalar(localtime(time_value));'
or
date -r time_value
Note that these commands will give date and time in your local time zone.
The Waikato University also collects traces using their DAG hardware. These are named differently to the NLANR traces. The DAG traces start with a three letter code identifying the trace, followed by -dag- identifying them as DAG traces, followed by the date and time of the trace, followed by the interface.
For example ACK-dag-19990708-121553-0-160000-161000.crl was collected at The University of Auckland at 12:15 on July 8, 1999 on interface 0 of the monitor.
The CoralReef system comes with some predefined applications that allow some analysis to be done on traces or on data collected, in real time, from a monitor. However, it is designed to make it easy for users to write their own applications and hopefully, contribute them back to CAIDA for inclusion in later releases.
In this exercise you will begin by studying a simple example program. Once you have understood that program you can move on to building your own CoralReef applications.
An example of a simple program that uses the CoralReef C library API is included as . The program opens the coral source(s) that are given on its command line and then prints the ATM header fields for each cell in the source. The following sections refer to that program.
The main body of the program is the while loop in lines 32-44. This loop repeats until either an error occurs, the duration, (i.e. time limit) specified on the command line is reached or, if input is from a trace, the trace is complete.
Within the loop a cell is read from the source at line 33. The call
coral_read_cell_all() extracts the next cell from the trace and
returns pointers to two data structures. The first structure describes
the block that the cell comes from and the second is the cell itself.
The third
parameter is for use with a real time monitor and specifies the
maximum time to wait for a cell. Full details are in the CoralReef
The program begins by setting up a signal handler (lines 22), so that sending a SIGINT signal (e.g. by typing control-C) will stop the program2.
It then parses any CoralReef arguments, i.e.those passed as a string following a -C. CoralReef applications have a set of common arguments that are processed by the CoralReef library. These include the source(s), maximum analysis duration, real time operating mode etc. The standard arguments are processed by the coral_config_arguments call at line 25.
The only CoralReef argument used by this program is duration, which specifies the time (in seconds) the program is to run (either live or using timestamps from a trace file). A program can set duration; on line 24 we set it to zero, which tells CoralReef it may allow the user to specify the duration actually required.
Before data can be fetched from a trace or a real time monitor the source must be opened and, for the case of a real time source, the the interface card(s) started. In the example program this is done by the calls coral_open_all() and coral_start_all() (at lines 27-30) which open and start all the sources that were given on the command line. If the source is a trace coral_start_all() has no effect but including it in the code means that the code can be used with either a trace or a real time source.
There are several events that can cause the main while loop to end. These are:
When analysis is complete the real time interface cards should be stopped and
the sources closed (this is done with coral_stop_all() and
coral_close_all() in the example program).
Like
coral_start_all(), coral_close_all() has no effect on a
trace file source.
The action of the main loop (lines 36-43) is to print the fields from the ATM header for each cell. coral_cell_header() returns a pointer to the cell's ATM header, the bytes of which are in network byte order. They are reordered into host byte order by the call on ntohs() on line 37. CoralReef provides a set of macros for unpacking an ATM header, these are used in the printf() statement on lines 39-42.
Remember that the network byte order (most significant byte first) might not match that of the machine that your program is running on. You will need to convert network byte order to your local host byte order. There is a macro on most Unix systems called ntohs() (network to host short) that does this conversion for a short integer. There are other similar macros for other conversions (see man for details).
It is easy to make simple mistakes that invalidate your analysis in an exercise of this type. For example if you are a byte out when testing for the TCP protocol value in the IP packet you will still get an answer but it will be meaningless. You will need to take care to avoid problems of this type. For example you can check that the values of the IP protocol field are mostly TCP (6) or UDP (17).
Run your program on the traces.
You probably tested every cell in the trace in section * above. This is appropriate on these traces because they only contain the first cell of each packet. Other traces contain the first two, the first and the last or or all cells in the trace. For these traces you would need to identify the first cell in the trace and just test that. Other traces are not cell-based at all.
The processing to do this is different depending on what range of cells are included in the trace. Your program would need to discover this and then behave appropriately. Your program would also fail on traces that were collected from non-ATM sources such as packet over sonnet (PoS) connections.
As an alternative to the cell interface CoralReef also includes a packet interface. This is described in the CoralReef .
gives an example of a simple program using the packet interface. In many respects it is similar to the program in appendix A. The most significant change is that the processing of packets (not cells) is done using a callback function (pkthandler) rather than by a loop in the mainline. The call coral_read_pkts at line 46 reads from the source calling pkthandler each time it assembles a complete packet from the interface iface.
The C API of CoralReef allows simple C programs analyse traces or data from a live monitor. In these exercise we have introduced the C API and undertaken a simple analysis. Hopefully this has made it clear how more compile analysis can be done.
You might like to think about how you would code some of the .
1 #include <stdio.h>
2 #include <signal.h>
3
4 #include "libcoral.h"
5
6 int done = 0;
7
8 static void quit(int arg) {
9 done = 1;
10 }
11
12 int main(int argc, char *argv[])
13 {
14 coral_atm_cell_t *cellp;
15 coral_iface_t *ifacep;
16 coral_blk_info_t *binfo;
17
18 const union atm_hdr *ahp;
19 union atm_hdr atm;
20 unsigned int vpvc;
21
22 signal(SIGINT, quit);
23
24 coral_set_duration(0);
25 if (coral_config_arguments(argc, argv) < 0)
26 exit(-1);
27 if (coral_open_all() <= 0)
28 exit(-1);
29 if (coral_start_all() < 0)
30 exit(-1);
31
32 while (!done) {
33 ifacep = coral_read_cell_all(&binfo, &cellp, NULL);
34 if (!ifacep) break;
35
36 ahp = coral_cell_header(ifacep, cellp);
37 atm.ui = ntohl(ahp->ui);
38 vpvc = atm.h.vpvc;
39 printf("gfc %02x, vpi:vci %02x:%04x, "
40 "oam_rm %1d congestion %1d sdu %1d clp %1d\n",
41 atm.h.gfc, get_vpvc_vp(vpvc), get_vpvc_vc(vpvc),
42 atm.h.oam_rm, atm.h.congestion, atm.h.sdu_type, atm.h.clp
43 );
44 } /* while !done */
45
46 coral_stop_all();
47 coral_close_all();
48
49 return 0;
50 }
1 #include <stdio.h>
2 #include <signal.h>
3
4 #include "libcoral.h"
5
6 static void quit(int arg) {
7 coral_stop_all();
8 coral_close_all();
9 exit(arg);
10 }
11
12 void pkthandler(
13 coral_iface_t *ifacep, const coral_timestamp_t *timestamp,
14 void *mydata, coral_pkt_buffer_t *packet,
15 coral_pkt_buffer_t *header, coral_pkt_buffer_t *trailer)
16 {
17 int z;
18 printf("interfacef %d:", coral_interface_get_number(ifacep));
19 for ( z = 0; z < packet->caplen; ++z) {
20 if ( z % 16 == 0 ) printf("\n ");
21 printf("%02x ", (unsigned char) packet->buf[z]);
22 }
23 printf("\n\n");
24 }
25
26 int main(int argc, char *argv[])
27 {
28 signal(SIGINT, quit);
29
30 coral_set_duration(0);
31 if (coral_config_arguments(argc, argv) < 0) exit(-1);
32 if (coral_open_all() <= 0) exit(-1);
33 if (coral_start_all() < 0) exit(-1);
34
35 coral_set_options(0, CORAL_OPT_PARTIAL_PKT);
36
37 coral_read_pkts(NULL, NULL, pkthandler, NULL, NULL, NULL, NULL);
38 quit(0);
39 }
Note: The indented lines in the file below start with four blanks. To work properly as a Makefile, each group of four blanks will need to be replaced with a TAB (control-I).
CORAL= /usr/local/Coral
# Directory containing CoralReef include and lib directories
CFLAGS= -g
# Source line numbers for dbg
all:
cells: cells.c
cc $(CFLAGS) -c -o cells.o -I$(CORAL)/include cells.c
cc $(CFLAGS) -o cells cells.o -L$(CORAL)/lib -lcoral -lz -lpcap
packets: packets.c
cc $(CFLAGS) -c -o packets.o -I$(CORAL)/include packets.c
cc $(CFLAGS) -o packets packets.o -L$(CORAL)/lib -lcoral -lz -lpcap
clean:
rm cells cells.o packets packets.o
1Why 48 bytes?
2Unix supports software signals.
A signal is very like a hardware interrupt in that it causes the
program to stop executing the code that it is currently processing and
to branch to the appropriate signal function. When the signal function
is complete execution returns to the code that was being executed
before the signal occurred. There are different types of signal for
different events. These include SIGINT which is sent when the program
is interrupted with control-C, and SIGALRM which is sent when a timer
(previously set with the +alrm+ signal call) expires. Each
signal can have a different function associated with it or a default
action may occur. The +signal()+ system call associated a function
with a signal (see lines 22 and 23). In this case the routine
+quit+ is only associated with +SIGINT+.