| Raw Data Description |
click here to see a raw data sample(CTC collection 11/04/97 1:20 p.m.)
DOS oc3mon output format:
spawn telnet 204.147.132.135 22
Trying 204.147.132.135...
Connected to 204.147.132.135.
Escape character is '^]'.
unfiltered output from expect script used to poll monitor
interface 0 IP bytes: 49075756
interface 1 IP bytes: 67039648
interface 0 all packets: 205418
interface 1 all packets: 186887
How many bytes and packets in each direction counted since last poll
interface 0 non IP packets: 674
interface 1 non IP packets: 642
interface 0 IP optioned packets: 0
interface 1 IP optioned packets: 0
Counts of packets that are non-IP and counts of IP packets with 1+
IP options, in each direction, seen since last poll
interface 0 first IP fragments: 0
interface 1 first IP fragments: 0
Counts of IP fragments where its the 1st fragment in the packet
interface 0 non-first IP fragments: 0
interface 1 non-first IP fragments: 0
Counts of IP fragments where its not the 1st fragment in the packet
flow allocation failures: 0
port flow allocation failures: 0
net flow allocation failures: 0
host flow allocation failures: 0
various allocation failures, any non-zero count is bad news, and
means you failed a mem alloc
following statistics were calculated at flow expiration time: (first packet >1 second after prev flow expiration)
This means that the following stats were (re)calculated upon a flow
expiring. The monitor checks for flows expiring about once every second. This
does not mean that this data is only from expired flows (that's later)
pending flows: 1506 max, 1411 avg
max# of un-expired flows in any second, and avg
new flows per second: 29 max, 12 avg
max# of new flows in any second, and avg
interface 0 packets per second: 1140 max, 701 avg
interface 1 packets per second: 1010 max, 638 avg
interface 0 bits per second: 3398888 max, 1340649 avg
interface 1 bits per second: 3441616 max, 1831385 avg
interface 0 avg bytes per packet: 535 max, 236 avg
interface 1 avg bytes per packet: 590 max, 363 avg
all pretty-much self-explanatory
IP_PREC 0:376321 6:15869
for x:y, #packets (y) having IP precedence field of value (x). Note
this 3-bit field is in the TOS byte and is supposedly "unused".
ST |packets| |avg. duration| |flows| |bytes| |total sec.s| and a bunch of
other stuff I can't remember but will fill in later:)
Here's a table describing all fields:
Key (for lines starting with SP or SB):
Each SB line summarizes data on a per-application basis. The example line
above shows all data collected for web server expired flows. See comments
on SP lines above for format.
NOTE - This and all AS-related data are currently programmed in our monitors
to "dump" out only once per hour. Most of our raw data files therefore have
none of this data.
SNB |src-AS#| |dest AS#| |flows| |packets| |bytes| etc.
interface 0 largest packet: 1500 bytes
interface 1 largest packet: 1500 bytes
largest packet seen since last poll in each direction
IP_LEN range=1 28:8 32:2412 34:4 36:93114 38:2 40:13341 41:2165
...and so on...
This is dump of the packet length array. The range is shown - if
1, it means these are exact packet length counts. The x:y format reports that
y packets were seen of length x. Note that only non-zero packet counts are printed.
(e.g. no packets of length 29, 30, or 31 were ever seen in the sample above).
PKT_FLOW 1:834 2:824 3:596 4:242 5:161 6:165 7:104
...and so on...
This is a dump of another array on packets per flow. For the x:y
format, y flows had x packets. These are based on expired flows only.
following statistics were calculated at query time: (now)
ST 330221 292.848 3832 321367 89607518 154123 13 1097 305987 1.0 1.0 1.0 1.0 83.864 23384 40 278.832 3230999 187795117971.527 95065 2814493518902.000 285838935999.789 1172685571.064
>From here on, we see everything based on expired flows only. The
ST line is a totals line, as follows:
SP [SB] lines columns:
Prot [sport dport] flows packets bytes dur %f %p %b %d fps
pps bps ppf bpf spf bpp vp vb vd copb cobd copd
SP 1 246 660 54196 2281 0.064 0.002 0.001 0.015
0.840 2 185 2.683 220 9.274 82 11.973 514725.325
405.041 547868.000 793859.912 9459.676
...and so on for several or more SP lines...
Each SP line summarizes data on an IP protocol. The format of SP
(and later SB) lines is shown in the output. Prot is IP protocol number, dur
is duration in cumulative seconds. The % field are percentages of total traffic
for flows, packets, and bytes. The v-fields are variances.
Column Number
1
2
3
4
5
Value
Protocol
Source/ Destination Port
Flows
Packets
Bytes
6
7
8
9
10
11
Duration (in seconds)
% of total flows
% of total packets
% of total bytes
% of total duration
flows/ second
12
v13
14
15
16
17
packets/ seconds
bytes /flow
seconds /flow
bytes /packet
variance in packets /flow
variance in bytes /flow
18
19
20
21
variance in seconds /flow
covariance in packet /flow [packets /flow] vs bytes /flow
covariance in bytes /flow vs seconds /flow
covariance in packets /flow vs seconds /flow
SS 20:26 21:18 23:24 25:97 53:463 79:6 80:702 113:29 119:20
123:142 161:198 1027:7 1166:6 2000:6 3128:54 6667:226 7000:28 7001:114
2 7003:1114 9875:32 17475:7 21061:6 22347:6 23457:13 24677:5
26407:8 27500:8 30271:5 30555:9 31613:7 51483:6 53241:5 61001:5 62049:7 62244:7
65441:5
SS: Ports with excessive references in pairings: line with one field
per port whose reference count exceeds the threshold, fields separated by tabs:
port number ":" reference count
SS port1:ref1 port2:ref2 port3:ref3
SC 5 36 4449 318 155 481
SC: port stats:
#ports #refs #TCP<1000 #UDP<1000 #ports<1000
port reference count threshold (settable by command line parm)
#ports exceeding above threshold (how many were in "SS" line)
sum of reference counts of all ports that exceeded the above threshold
count of TCP port triples
count of UDP port triples
count of all port triples
SB 6 80 0 305 5449 3082243 2822.861 0.080
0.017 0.034 0.018 1.041 19 10525 17.866 10106 9.255 566
2306.972 709437949.882 215.355 425323601.000 56156751.848
...and so on for what may be *many* SB lines...
one line for each port pair exceeding thresholds for packets, bytes,
or flows (which default to .5% of total seen on links); fields separated by
tabs
SHB 128.253.22.203 6 80 8 65 64889 75.184 0.002
0.000 0.001 0.000 0.027 0 222 8.125 8111 9.398 998
102.982 176789460.982 108.963 1468402.000 1154547.173 1064.939
...and so on for what may be *many* SHB lines...
Each SHB line shows a "heavy hitter"; our monitors are programmed
now to track the heaviest traffic sources of web and ftp-data. The format is:
SHB |ip address| |IP proto| |source port| |flows| |packets| |bytes| etc...
SHB: per (host pair, port pair) for highest volume (source) hosts line
per host triple exceeding threshold for packets, bytes, or flows (which default
to .5% of total seen on links); fields separated by tabs:
source IP address,
IP protocol
source TCP/UDP port
the usual stuff (see definition of "SP" line above)
SNT 326625 3808.165 45546 6980022 2496067595 2453331 12
1833 655452 1.0 1.0 1.0 1.0 153.252 54803 54 357.602
44460800 5580775480173.786 139524 514974801457264.000 70498244
61178.270 20679887972.104
SNT: same as "ST" except it covers the period since network stats were
last cleared (defaults to one hour), and count of src/dest IP address routing
table lookup failures
191/0 source/destination net lookup failures
AS-related failure: how many IP address -to- AS number maps failed
SNS 1:583 9:1585 16:988 17:40 22:261 25:20 26:39499
...and so on...all on one line
SNS: line with one field per port whose reference count exceeds the threshold,
fields separated by tabs: Autonomous System (AS) number ":" reference count
SNC 0 127 101918 179
SNC: line with these fields separated by tabs:
AS# reference count threshold (settable by command line parm)
#AS's exceeding threshold (how many were in "SNS" line),
sum of reference counts of all AS#'s that exceeded the above threshold count
of all AS# pairs
SNM 64533='577 {549,239}'
SNMone line per AS# created for an aggregation, with fields separated
by spaces: AS# created, "=", list of all AS#'s in the aggregate, like "{1,2,3,{5,6}}"
SNB 685 32768 21 15745 5620564 15043.987 0.000 0.003
0.003 0.007 0.006 4 1477 749.762 267646 716.380 357 3092030.
290 666903614687.090 1829978.785 31118249931.000 17362084211.876
50329491.550
...and so on...perhaps many SNB lines
SNB lines show AS pairs. The format is:
SNB one line per AS pair that exceeds any of the thresholds for packets,
bytes, or flows (which default to .5% of total seen on links); fields separated
by tabs: src AS#, dest AS#, usual stuff (see definition of "SP" line above).
0 flows, 0 packets, 0 bytes, 0.0 duration ignored by network report
This is a summary line for AS reporting so we know how much data
was "below" the thresholds for keeping around in AS stats keeping, and thus
were ignored.
TCP flows: 1153
TCP packets: 28395
TCP raw forward bytes (including IP): 8391466
TCP good forward bytes (not including IP or TCP or options): 7111633
TCP good backward bytes (not including IP or TCP or options): 8624971
TCP small leftover hole flows: 3
TCP small leftover hole bytes: 29599
TCP small leftover hole packets: 87
TCP big leftover hole flows: 0
TCP big leftover hole bytes: 0
TCP big leftover hole packets: 0
TCP count of flows for each multiple hole count: 0:1148 1:1 4:1
9:1 16:1 22:1
TCP count of flows for each single hole count: 0:1133 1:15 2:3 3:1
7:1
TCP count of flows for each count of packets in single holes: 0:1150 1:3
TCP count of flows for each count of backtracking packets: 0:1148 4:1
11:1 28:1 48:1 70:1
TCP count of flows for each count of packets with duplicate forward bytes:
0:888 1:129 2:28 3:80 4:1 5:2 6:1 7:6 8:2 10:2
11:2 13:2 14:1 18:1 20:1 21:2 22:1 27:1 33:1 39:1
255:1
TCP count of flows for each count of packets with duplicate backward bytes:
0:394 1:369 2:220 3:67 4:34 5:10 6:6 7:5 8:3 9:4
10:3 11:3 12:3 13:1 14:1 15:1 16:2 19:3 20:1 23:1
24:1 28:1 29:1 33:2 34:1 35:1 37:1 40:1 45:1 50:1
53:1 54:1 60:1 66:1 74:1 85:1 86:1 93:1 158:1 255:2
TCP count of flows for each count of packets with bad TCP header length:
0:1153
TCP count of flows for each combination of TCP flags: (2)S:136 (4)R:16
(6)RS:2 (16)A:29 (17)AF:10 (18)AS:9 (19)ASF:32 (20)AR:2
8 (23)ARSF:2 (24)AP:22 (25)APF:24 (26)APS:60 (27)APSF
:581 (28)APR:5 (30)APRS:171 (31)APRSF:26
TCP count of flows for each log2 of max RX window size: -1:46 9:43 10:3
11:2 12:136 13:592 14:135 15:196
Lots of TCP instrumentation that deserve more detail here :)
seconds since last flow housecleaning: 0.049502 min, 1.953717 max, 2.050762 avg
deleted flows: 44 max, 12.7 avg
2.92 seconds for previous query socket to drain through network
previous flows_query() elapsed time was 151.103 milliseconds
BufferQueue fullness 0 min, 1 max, 0.000003 avg over 10933695 iterations
Connection closed by foreign host.
Some housecleaning details on the monitor