Sniffer configuration

From VoIPmonitor.org
Revision as of 17:12, 24 July 2013 by Festr (talk | contribs)
Jump to navigation Jump to search

Configuration file has only one section named [general] where all configuration directives belongs. List of directives will now follow with their description and recommendation values. Name in [ ] brackets is equivalent for command line which takes precendence over configuration file. in case of running more voipmonitor instances on the same or another servers configured to save to one database and the same cdr table it is possible to differentiate CDR by id_sensor column. If you set id_sensor >= 0 the number will be saved in cdr.id_sensor column.

id_sensor

This specifies on which interface will voipmonitor listen. It can listen on one interface or on all interfaces. To listen on all interfaces use interface = any

id_sensor = 1

interface

listening interface. Can be 'any' which will listen on all interfaces - NOTE that "any" will not put intefaces into promiscuous mode and you have to do it with "ifconfig eth0 promisc" check if you are not using -i ethX argument in command line as it has more priority than this configuration file

interface = eth0  [ -i ]

mirror

since version 8 sniffer implements new mirroring option. Sender is packing data to compressed stream over the TCP to remote sniffer. if you are going to use this sniffer only as a mirroring sniffer all you need is to set interface, packetbuffer_* set compression on and set packetbuffer_file_* so in case the connection to remote sniffer will die or will be temporarily slow the sender will not loose single packet. The mirroring is trying to reconnect in case of failure. Packets are mirrored including the the original timestamp and headers. this mirroring hopefully replaces pcapscandir feature which will be probably removed in favor of this approach.

The sender needs to set only interface, ringbuffer, packetbuffer_*, filter and this two folling mirror_destination_*

mirror_destination_ip          = 
mirror_destination_port        =

On receiver set mirror_bind_ip and mirror_bind_port. Do not forget to set firewall so no other except the sender will be able to connect to the receiver

mirror_bind_ip               = 
mirror_bind_port             =

scanpcapdir

scan pcap files folder and read file by file. This is in conjunction with running tcpdump which creates pcap file each 5 seconds (-G 5) storing pcap files named by UNIX_TIMESTAMP to /dev/shm/voipmonitor folder (do not forget create it) using 1GB ring buffer to avoid losing packets (-B500000 - you can lower it but not higher) filtering udp packets (udp parameter whcih you can change to your needs). voipmonitor then reads created files (and delete it after processing. This approach can be used for testing throughput or for very high voip traffic (>500Mbit). If the sniffer is able to process pcap files in realtime - there will be in /dev/shm/voipmonitor folder only one or two pcap files. If the sniffer is not able to process in realtime (blocking by I/O or by CPU) number of pcap files will grow faster then the sniffer is able process.

scanpcapdir = /dev/shm/voipmonitor

WARNING: libpcap < 1.1 contains memory leak when pcap filter is set - do not set filter in this config or upgrade libpcap to the latest (debian 6 libpcap contains the leak) static compiled voipmonitor from voipmonitor.org contains the latest libpcap

tcpdump example command:
nice -n -20 tcpdump -B500000 -i eth2 udp -G 5 -w /dev/shm/voipmonitor 2>/dev/null 1>/dev/null & 

ringbuffer

ringbuffer is circular memory queue directly in kernel memory space. libpcap is reading from this queue and delivers packets to voipmonitor. If the network rate is > 100 Mbit we recommend to set ringbuffer to at least 500. Maximum value is 2000 MB. ringbuffer = 50 [ --ring-buffer ]

packetbuffer

packet buffer is new voipmonitor buffering architecture (since version 8). If enabled new threads are created which raads packets from kernel ringbuffer and queues them into dynamically allocated memory. Packets are dequeued and passed to next threads which reads the content. This will ensure that kernel ringbuffer will not overrun due to CPU or disk I/O spikes. packet buffer will dynamically grow until packetbuffer_total_maxheap is reached. Compression can be enabled (packetbuffer_compress) which compress the buffer with fast snappy algorythm with 50% compression ratio thus doubling the time when the buffer gets filled (600Mbit traffic consumes ~30% one one core Xeon E5-2620).

It is also possible to use disk buffer if the packet buffer memory is filled by enabling packetbuffer_file_totalmaxsize which is usefull when sniffer is only mirroring data over TCP to another sniffer - if the connection brakes or slowed down and packet buffer gets filled it will start using file buffer until the connection reestablishes so no single packet is lost. Enabling file buffer in non mirroring mode to the same disk as spooldir will get not much benefit because if the process is blocked mainly due to disk I/O it has no benefit to add more I/O by caching unprocessed packets to the same I/O layer.

packetbuffer_enable             = yes packetbuffer_total_maxheap      = 2000 #in MB  packetbuffer_compress           = no #enable compression if you have >2 CPU cores   packetbuffer_file_totalmaxsize  = 0 #MB. Default is disabled.  packetbuffer_file_path          = /var/spool/voipmonitor/packetbuffer

rtpthreads

number of threads to process RTP packets. If not specified it will be number of available CPUs - 1. If equal to zero RTP threading is turned off. Each thread allocates default 20MB for buffers (increase to 100 on very high loads). This buffer can be controlled with rtpthread-buffer. For < 150 concurrent calls you can turn it off.

rtpthreads = 3

natalias

in case the SIP(media) server is behind public IP (1.1.1.1) NATed to private IP (10.0.0.3) to sniff all traffic correctly you can specify alias for this case. You can specify more netaliases duplicating rows. In most cases this is not necessary because voipmonitor is able to track both RTP streams based on the other side IP. But if the stream is incoming from another IP then SIP source signalization and also from another IP than the SIP device which is also behind NAT its impossible to track the correct IP. Please note that this is for case where the SIP server is behind NAT and also the client is behind NAT. If your SIP server has public IP do not bother with this.

natalias = 1.1.1.1 10.0.0.3 
natalias = 1.1.1.2 10.0.0.4


managerip

define bind address for manager interface. Default is 127.0.0.1 it is not recommended to change this unless really needed due to security. If you need it on some other IP make sure you set firewall and change the standard port for better security

managerip = 127.0.0.1

managerport

This specifies TCP port which will voipmonitor listen for incoming connections which controls voipmonitor or for getting information about calls. *reload configuration echo reload | nc localhost 5029 *get number of calls echo totalcalls | nc localhost 5029 *get list of calls in json format echo listcalls | nc localhost 5029

managerport = 5029 [ --manager-port <port number> ]

managerclient

connects to server and listen for commands.

managerclient = serverip or hostname
managerclientport = 1234

sipport

define SIP ports wihch will voipmonitor listen. For each port make new line with sipport = port (multiple lines)

sipport = 5060
sipport = 5061 sipport = 5062

rtptimeout

rtptimeout is important value which specifies how much seconds from the last SIP packet or RTP packet is call closed and writen to database. It means that if you need to monitor ONLY SIP you have to set this to at leat 2 hours = 7200 assuming your calls is not longer than 2 hours. Take in mind that seting this to very large value will cause to keep call in memory in case the call lost BYE and can consume all memory and slows down the sniffer - so do not set it to very high numbers. Default is 300 seconds.

rtptimeout = 300

jitterbuffer

By default voipmonitor uses three types of jitterbuffer simulators to compute MOS score. First variant is saved into cdr.[ab]_f1 and represents MOS score for devices which has only fixed 50ms jitterbuffer. Second variant is same as first but for fixed 200ms and is saved to cdr.[ab]_f2 Third varinat is adaptive jitterbuffer simulator up to 500ms Jitterbuffer simulator is the most CPU intensive task which is voipmonitor doing. If you are hitting CPU 100% turn off some of the jitterbuffer simulator. Recommended for higher loads is to use only fixed 200ms (f1)

jitterbuffer_f1 = yes
jitterbuffer_f2 = yes
jitterbuffer_adapt = yes

callslimit

callslimit will limit maximum numbers of calls processed by voipmonitor at the same time. If calls are over limit it will be ignored (INVITE)

callslimit = 0

rtp-firstleg

this is important option if voipmonitor is sniffing on SIP proxy like kamailio or openser and sees both RTP leg of CALL. In that case use this option. It will analyze RTP only for the first LEG and not each 4 RTP streams which will confuse voipmonitor. Drawback of this switch is that voipmonitor will analyze SDP only for SIP packets which have the same IP and port of the first INVITE source IP and port. It means it will not work in case where phone sends INVITE from a.b.c.d:1024 and SIP proxy replies to a.b.c.d:5060.

rtp-firstleg = no [ --rtp-firstleg ]

deduplicate

duplicate check do md5 sum for each packet and if md5 is same as previous packet it will discard it. WARNING: md5 is expensive function (slows voipmonitor 3 times) so use it only if you have enough CPU or for pcap conversion only. Default is no.

deduplicate = yes

sipoverlap

enable/disable updating called number from To: header from each caller INVITE. Default is enabled so it supports overlap dialing (RFC 3578)if you want to disable this behaviour and see always number only from the first INVITE set sipoverlap = no

sipoverlap = yes


sip-register

Enable parsing of SIP REGISTER message. SQL register table stores active SIP registrations. Once it expires it is removed from the table to new sql register_state table. The register state table is used to store changes in registrations. SQL table register_failed is used to store all failed sip register. To not overload this table there is counter column which adds +1 for each failed register from the same source.

sip-register = no [ -R ]

sip-register-active-nologbin

if mysql binlog is enabled, skip binlog inserts into active table (which is MEMORY type) if you still want to replicate this too (huge I/O impact) set it to = no sip-register-active-nologbin = yes

nocdr

if yes, voipmonitor will not save CDR to MySQL

nocdr = no [ -c ]


savesip

Store SIP packets to pcap file.

savesip = [ --sip-register ]

savertp

save RTP packets to pcap file. savertp = yes automatically saves RTCP packets you can also save only RTP header without AUDIO: savertp = header if save RTP is aneblad it will also save UDPTL packets (used for T.38) you can also set savertp = no and control what calls will record RTP in mysql table filter_ip or filter_tel which is controled in GUI -> Capture rules. Sending reload command will reload configuration from filter_* table. You can also set savertp = yes but denies recording RTP based on rules in filter_* table.

savertp = yes | header [ -R ]

pcapsplit

voipmonitor by default splits SIP and RTP packets to individual files (in case spooldiroldschema = no) which are located in SIP and RTP directories. This feature allows instance cleaning RTP streams differently then SIP packets to join two pcap files SIP+RTP use mergecap command line utility which is included in wireshark package default = yes | spooldiroldschema must be set to no

pcapsplit = yes

savertcp

Store RTCP packets to pcap file.

savertcp = yes [ --save-rtcp ]

saveudptl

save UDPTL packets (T.38). If savertp = yes the udptl packets are saved automatically. If savertp = no and you want to save only udptl packets enable saveudptl = yes and savertp = no

saveudptl = yes

savegraph

This is usefull only if you have commercial WEB GUI which uses graph files for ploting graph

savegraph = plain [ -G or --save-graph=[gzip|plain] ]

saveaudio

save RTP payload to audio file. Choose 'wav' for WAV PCM or 'ogg' for OGG 25kbps format. please note that this has great impact on I/O and can overload your storage leading to lose packets. Better way is to store only sip+rtp and convert wav files on demand.

saveaudio = wav

keycheck

default path to WEB GUI used to construct path to key check for codecs

default paths: #keycheck = /var/www/voipmonitor/php/keycheck.php #keycheck = /var/www/html/voipmonitor/php/keycheck.php

saverfc2833

in case you are not saving RTP at all but you still want to save DTMF carried over RTP packets (RFC2833) you can enable this option. This feature slows down a bit processing RTP packets in main read thread in casse voipmonitor runs in threads. default = 0

saverfc2833 = 0

dtmf2db

Enable storing DTMF (SIP INFO or RFC2833) to cdr_dtmf database. It will store DTMF time and key then it will be shown in SIP history in the GUI

dtmf2db = 0

norecord-header

if any of SIP message during the call contains header X-VoipMonitor-norecord call will be not converted to wav and pcap file will be deleted.

norecord-header = yes

= norecord-dtmf =if any of SIP message during the call contains DTMF INFO sequence "*0" call will be not converted to wav and pcap file will be deleted. default: disabled

norecord-dtmf = yes

pauserecordingdtmf

enable pausing RTP/WAV recording if DTMF sequence detected. default: disabled

pauserecordingdtmf = *9

dumpallpackets

dump all packets to /tmp/voipmonitor-[UNIX_TIMESTAMP].pcap

dumpallpackets = yes

mos_g729

enable MOS score for G.729 codec. If enabled, all cdr with 0 packet loss and stable delays will have maximum MOS of 3.92 and for loss and unstable delay MOS will be calculated according to ITU-T objective PESQ method for G.729 codec. if you want to use MOS as good search value which corellates loss and delay into single value leave it disabled (which is by default). If set to no, all calls will be calculated like it is G.711. Recommended value = no

mos_g729 = no

custom_headers

Since 7.0RC7.

enable storing custom sip headers to database column cdr_next.custom_header_headername. You can specify more headers delimited by ";". WARNING - when you enable this feature voipmonitor will autoupgrade cdr_next table which can take hours depending on how large the table is. In GUI there is new section Settings#CDR_Custom_header.

custom_headers = X-asterisk-Info ; X-myheader

analogical for SIP message is custom_headers_message

custom_headers_message = X-asterisk-Info ; X-myheader

match_header

enable saving content of custom header (typicaly in-reply-to) to cdr_next.match_header this header is used in related CDR GUI for matching legs to onen call

match_header = in-reply-to

pcapcommand

pcapcommand will run command after pcap file is closed (after call ends). %pcap% is substitution for real pcap file name. execution is guaranteed to run in serialized way (not in parallel)WARNING - pcapcommand is implemented by forking program which is very expensive and is causing TLB shootouts on multicore system which can generate 500 000 interrupts / sec causing system to drop packets. Watch the performance carefuly (with "vmstat 1" column "in"). Gziping pcap files will be implemented as native function directly in C++ to obey TLB shootdowns.

pcapcommand = gzip %pcap%

filter

libpcap tcpdump style filter. Voipmonitor listens in default only for UDP packets. Unfortunatly filtering UDP packets will filter all VLAN tagged packets which means that you cannot filter only UDP if you want to listen to VLAN tagged packets.

WARNING - if you need to sniff IPinIP (like mirrored packets from voipmonitor) filter = udp will filter all those packets. In this case just disable filter.


filter = udp or (vlan and udp) [ -f ]

convertchar

list characters that should be converted to underscore (_) in filenames if you want to include space, put it between other characters, like ': :' (will convert ':' and ' ' to '_') defaults to nore

# example - avoid ':' when Call-Id contains port number convertchar = :

spooldir

This is directory where all pcap/graph/wav files are stored.

spooldir = /var/spool/voipmonitor [ -d ]

spooldiroldschema

new spooldir schema stores all files to year-mon-day/hour/minute/[ALL|SIP|RTP|AUDIO] directories if you need to have the old schema year-mon-day/* enable spooldiroldschema = yes. default = no

spooldiroldschema = no

cachedir

store pcap and graph file to <cache/dir> and move it after call ends to spool directory. Moving all files are guaranteed to be serialized which solves slow random write I/O on magnetic or other media. Typical cache directory is /dev/shm/voipmonitor which is in RAM and grows automatically or /mnt/ssd/voipmonitor which is mounted to SSD disk or some very fast SAS/SATA disk where spool can be network storage or raid5 etc. wav files are not implemented yet

cachedir = /dev/shm/voipmonitor

cleandatabase

cleandatabase removes database partitions for tables cdr, cdr_next, cdr_rtp and cdr_dtmf older than X days. Cleandatabase parameter represents number of days - if you set 90 it will remove partition 90 days old. If the sniffer will not run for several days it will not clean partitions which was not cleaned even you start it.

  • default is disabled
  • Added since 8.0RC6
cleandatabase = 0 (disabled) 


cleanspool_interval

clean spool directory every two hours from oldest file until spooldir size = cleanspool_size (in MB) WARNING - if you have milions files set cleanspool_interval to 24 hour (clean once per day) cleaning is very slow and can overload I/O (in seconds). Default is disabled.

cleanspool_interval = 7200

cleanspool_size

set target size of the spool directory for cleaning script (in MB). Default is disabled.


cleanspool_size = 50000
#usefull command to clean all RTP files older 7 days # find /var/spool/voipmonitor -maxdepth 1 -type d -mtime +7 -name '20*' | (while read d; do rm -rf $d/*/*/RTP; done)

promisc

This option is only relevant if you are mirroring traffic to your network card/cards. This will not work if interface = any - in this case, use ifconfig to put your desired interfaces to promis mode. Default value is yes and you want to turn it of on command line ues -n which will turn it off.

promisc = yes [ -n ]

database

sqldriver = mysql  #sqldriver = odbc #odbcdriver = mssql #odbsdsn = voipmonitor #odbcuser = root #odbcpass = 

voipmonitor can connect to mysql server or odbc driver. connecting voipmonitor to msssql please refer to README.mssql

mysqlhost = localhost [ -h ]

mysql server, default is localhost

mysqldb = voipmonitor [ -b ]

mysql database, default is voipmonitor

sqlcdrtable_last30d = cdr_last30d sqlcdrtable_last7d = cdr_last7d sqlcdrtable_last1d = cdr_last1d

enable redundant tables which WEB GUI use for speedup searches on giant CDR tables with millions of records. This is deprecated in favor of MySQL partitioning.

mysqlusername = root

mysql username, default is root

mysqlpassword =

mysql password, default is no password