Raw Data
Description

 

click here to see a raw data sample(CTC collection 11/04/97 1:20 p.m.)

 

DOS oc3mon output format:

spawn telnet 204.147.132.135 22 
Trying 204.147.132.135...
Connected to 204.147.132.135.
Escape character is '^]'.
unfiltered output from expect script used to poll monitor

interface 0 IP bytes: 49075756
interface 1 IP bytes: 67039648
interface 0 all packets: 205418
interface 1 all packets: 186887
How many bytes and packets in each direction counted since last poll

interface 0 non IP packets: 674
interface 1 non IP packets: 642
interface 0 IP optioned packets: 0
interface 1 IP optioned packets: 0
Counts of packets that are non-IP and counts of IP packets with 1+ IP options, in each direction, seen since last poll

interface 0 first IP fragments: 0
interface 1 first IP fragments: 0
Counts of IP fragments where its the 1st fragment in the packet

interface 0 non-first IP fragments: 0
interface 1 non-first IP fragments: 0
Counts of IP fragments where its not the 1st fragment in the packet

flow allocation failures: 0
port flow allocation failures: 0
net flow allocation failures: 0
host flow allocation failures: 0
various allocation failures, any non-zero count is bad news, and means you failed a mem alloc

following statistics were calculated at flow expiration time: (first packet >1 second after prev flow expiration)
This means that the following stats were (re)calculated upon a flow expiring. The monitor checks for flows expiring about once every second. This does not mean that this data is only from expired flows (that's later)

pending flows: 1506 max, 1411 avg
max# of un-expired flows in any second, and avg

new flows per second: 29 max, 12 avg
max# of new flows in any second, and avg

interface 0 packets per second: 1140 max, 701 avg
interface 1 packets per second: 1010 max, 638 avg
interface 0 bits per second: 3398888 max, 1340649 avg
interface 1 bits per second: 3441616 max, 1831385 avg
interface 0 avg bytes per packet: 535 max, 236 avg
interface 1 avg bytes per packet: 590 max, 363 avg
all pretty-much self-explanatory

IP_PREC 0:376321 6:15869
for x:y, #packets (y) having IP precedence field of value (x). Note this 3-bit field is in the TOS byte and is supposedly "unused".

interface 0 largest packet: 1500 bytes
interface 1 largest packet: 1500 bytes
largest packet seen since last poll in each direction

IP_LEN range=1 28:8 32:2412 34:4 36:93114 38:2 40:13341 41:2165
...and so on...
This is dump of the packet length array. The range is shown - if 1, it means these are exact packet length counts. The x:y format reports that y packets were seen of length x. Note that only non-zero packet counts are printed. (e.g. no packets of length 29, 30, or 31 were ever seen in the sample above).

PKT_FLOW 1:834 2:824 3:596 4:242 5:161 6:165 7:104
...and so on...
This is a dump of another array on packets per flow. For the x:y format, y flows had x packets. These are based on expired flows only.

following statistics were calculated at query time: (now)
ST      330221  292.848 3832    321367  89607518        154123  13      1097    305987  1.0     1.0     1.0     1.0     83.864  23384   40      278.832 3230999 187795117971.527        95065   2814493518902.000       285838935999.789        1172685571.064
>From here on, we see everything based on expired flows only. The ST line is a totals line, as follows:

ST |packets| |avg. duration| |flows| |bytes| |total sec.s| and a bunch of other stuff I can't remember but will fill in later:)

SP [SB] lines columns:
   Prot [sport dport] flows packets bytes dur %f %p %b %d fps
   pps bps ppf bpf spf bpp vp vb vd copb cobd copd
SP      1       246     660     54196   2281    0.064   0.002   0.001   0.015
0.840   2       185     2.683   220     9.274   82      11.973  514725.325
405.041 547868.000      793859.912      9459.676

...and so on for several or more SP lines...
Each SP line summarizes data on an IP protocol. The format of SP (and later SB) lines is shown in the output. Prot is IP protocol number, dur is duration in cumulative seconds. The % field are percentages of total traffic for flows, packets, and bytes. The v-fields are variances.

Here's a table describing all fields:


Key (for lines starting with SP or SB):

Column Number 1 2 3 4 5
Value Protocol Source/ Destination Port Flows Packets Bytes
6 7 8 9 10 11
Duration (in seconds) % of total flows % of total packets % of total bytes % of total duration flows/ second
12 v13 14 15 16 17
packets/ seconds bytes /flow seconds /flow bytes /packet variance in packets /flow variance in bytes /flow
18 19 20 21
variance in seconds /flow covariance in packet /flow [packets /flow] vs bytes /flow covariance in bytes /flow vs seconds /flow covariance in packets /flow vs seconds /flow


SS      20:26   21:18   23:24   25:97   53:463  79:6    80:702  113:29  119:20
123:142 161:198 1027:7  1166:6  2000:6  3128:54 6667:226        7000:28 7001:114
2       7003:1114       9875:32 17475:7 21061:6 22347:6 23457:13        24677:5
26407:8 27500:8 30271:5 30555:9 31613:7 51483:6 53241:5 61001:5 62049:7 62244:7
65441:5

SS: Ports with excessive references in pairings: line with one field per port whose reference count exceeds the threshold, fields separated by tabs: port number ":" reference count
 SS port1:ref1 port2:ref2 port3:ref3   

SC      5       36      4449    318     155     481

SC: port stats:
#ports #refs #TCP<1000 #UDP<1000 #ports<1000 

port reference count threshold (settable by command line parm)
#ports exceeding above threshold (how many were in "SS" line)
sum of reference counts of all ports that exceeded the above threshold
count of TCP port triples
count of UDP port triples
count of all port triples

SB      6       80      0       305     5449    3082243 2822.861        0.080
0.017   0.034   0.018   1.041   19      10525   17.866  10106   9.255   566
2306.972        709437949.882   215.355 425323601.000   56156751.848  

...and so on for what may be *many* SB lines...
one line for each port pair exceeding thresholds for packets, bytes, or flows (which default to .5% of total seen on links); fields separated by tabs

Each SB line summarizes data on a per-application basis. The example line above shows all data collected for web server expired flows. See comments on SP lines above for format.

SHB     128.253.22.203  6       80      8       65      64889   75.184  0.002
0.000   0.001   0.000   0.027   0       222     8.125   8111    9.398   998
102.982 176789460.982   108.963 1468402.000     1154547.173     1064.939

...and so on for what may be *many* SHB lines...

Each SHB line shows a "heavy hitter"; our monitors are programmed now to track the heaviest traffic sources of web and ftp-data. The format is:

SHB |ip address| |IP proto| |source port| |flows| |packets| |bytes| etc...


SHB: per (host pair, port pair) for highest volume (source) hosts line per host triple exceeding threshold for packets, bytes, or flows (which default to .5% of total seen on links); fields separated by tabs:
source IP address,
IP protocol
source TCP/UDP port
the usual stuff (see definition of "SP" line above)

SNT     326625  3808.165        45546   6980022 2496067595      2453331 12
1833    655452  1.0     1.0     1.0     1.0     153.252 54803   54      357.602
44460800        5580775480173.786       139524  514974801457264.000     70498244
61178.270       20679887972.104

SNT: same as "ST" except it covers the period since network stats were last cleared (defaults to one hour), and count of src/dest IP address routing table lookup failures

191/0 source/destination net lookup failures
AS-related failure: how many IP address -to- AS number maps failed

NOTE - This and all AS-related data are currently programmed in our monitors to "dump" out only once per hour. Most of our raw data files therefore have none of this data.

SNS     1:583   9:1585  16:988  17:40   22:261  25:20   26:39499
...and so on...all on one line

SNS: line with one field per port whose reference count exceeds the threshold, fields separated by tabs: Autonomous System (AS) number ":" reference count

SNC     0       127     101918  179

SNC: line with these fields separated by tabs:
AS# reference count threshold (settable by command line parm)
#AS's exceeding threshold (how many were in "SNS" line),
sum of reference counts of all AS#'s that exceeded the above threshold count of all AS# pairs

SNM 64533='577 {549,239}'

SNMone line per AS# created for an aggregation, with fields separated by spaces: AS# created, "=", list of all AS#'s in the aggregate, like "{1,2,3,{5,6}}"

SNB     685     32768   21      15745   5620564 15043.987       0.000   0.003
0.003   0.007   0.006   4       1477    749.762 267646  716.380 357     3092030.
290     666903614687.090        1829978.785     31118249931.000 17362084211.876
50329491.550

...and so on...perhaps many SNB lines
SNB lines show AS pairs. The format is:

SNB |src-AS#| |dest AS#| |flows| |packets| |bytes| etc.
SNB one line per AS pair that exceeds any of the thresholds for packets, bytes, or flows (which default to .5% of total seen on links); fields separated by tabs: src AS#, dest AS#, usual stuff (see definition of "SP" line above).

0 flows, 0 packets, 0 bytes, 0.0 duration ignored by network report
This is a summary line for AS reporting so we know how much data was "below" the thresholds for keeping around in AS stats keeping, and thus were ignored.

TCP flows: 1153
TCP packets: 28395
TCP raw forward bytes (including IP): 8391466
TCP good forward bytes (not including IP or TCP or options): 7111633
TCP good backward bytes (not including IP or TCP or options): 8624971
TCP small leftover hole flows: 3
TCP small leftover hole bytes: 29599
TCP small leftover hole packets: 87
TCP big leftover hole flows: 0
TCP big leftover hole bytes: 0
TCP big leftover hole packets: 0
TCP count of flows for each multiple hole count:        0:1148  1:1     4:1
9:1     16:1    22:1
TCP count of flows for each single hole count:  0:1133  1:15    2:3     3:1
7:1
TCP count of flows for each count of packets in single holes:   0:1150  1:3
TCP count of flows for each count of backtracking packets:      0:1148  4:1
11:1    28:1    48:1    70:1
TCP count of flows for each count of packets with duplicate forward bytes:
0:888   1:129   2:28    3:80    4:1     5:2     6:1     7:6     8:2     10:2
11:2    13:2    14:1    18:1    20:1    21:2    22:1    27:1    33:1    39:1
255:1
TCP count of flows for each count of packets with duplicate backward bytes:
0:394   1:369   2:220   3:67    4:34    5:10    6:6     7:5     8:3     9:4
10:3    11:3    12:3    13:1    14:1    15:1    16:2    19:3    20:1    23:1
24:1    28:1    29:1    33:2    34:1    35:1    37:1    40:1    45:1    50:1
53:1    54:1    60:1    66:1    74:1    85:1    86:1    93:1    158:1   255:2
TCP count of flows for each count of packets with bad TCP header length:
0:1153
TCP count of flows for each combination of TCP flags:   (2)S:136        (4)R:16
(6)RS:2 (16)A:29        (17)AF:10       (18)AS:9        (19)ASF:32      (20)AR:2
8       (23)ARSF:2      (24)AP:22       (25)APF:24      (26)APS:60      (27)APSF
:581    (28)APR:5       (30)APRS:171    (31)APRSF:26
TCP count of flows for each log2 of max RX window size: -1:46   9:43    10:3
11:2    12:136  13:592  14:135  15:196
Lots of TCP instrumentation that deserve more detail here :)

seconds since last flow housecleaning: 0.049502 min, 1.953717 max, 2.050762 avg
deleted flows: 44 max, 12.7 avg
2.92 seconds for previous query socket to drain through network
previous flows_query() elapsed time was 151.103 milliseconds
BufferQueue fullness 0 min, 1 max, 0.000003 avg over 10933695 iterations
Connection closed by foreign host.
Some housecleaning details on the monitor