Difference between revisions of "Data Cleaning"
| (2 intermediate revisions by one other user not shown) | |||
| Line 3: | Line 3: | ||
By default sniffer stores all data to /var/spool/voipmonitor which can be changed in voipmonitor.conf - spooldir = ... | By default sniffer stores all data to /var/spool/voipmonitor which can be changed in voipmonitor.conf - spooldir = ... | ||
| − | Version >= 8.0 implements new cleaning in voipmonitor.conf. Cleaning procedure runs every | + | Version >= 8.0 implements new cleaning in voipmonitor.conf. Cleaning procedure runs every 5 minutes and checks the size or days according to following options. If you set maxpoolsize it will wipe out the oldest data every hour until the size is reached. maxpooldays keeps maximum number of data to set days. The same is for sip rtp and graph so you can keep sip pcaps longer than rtp pcaps. All following options can be activated at onc and all sizes are in MB |
#set default maxpoolsize to 100 GB (102400 MB) | #set default maxpoolsize to 100 GB (102400 MB) | ||
| Line 17: | Line 17: | ||
maxpoolgraphsize = | maxpoolgraphsize = | ||
maxpoolgraphdays = | maxpoolgraphdays = | ||
| + | |||
| + | maxpoolaudiosize = | ||
| + | maxpoolaudiodays = | ||
Detailed description: | Detailed description: | ||
| Line 30: | Line 33: | ||
= Database cleaning = | = Database cleaning = | ||
| − | Since version 7 the sniffer is partitioning tables by days which allows efficient cleaning of partitions. There is configuration option in voipmonitor.conf [Sniffer_configuration#cleandatabase]. | + | Since version 7 the sniffer is partitioning tables by days which allows efficient cleaning of partitions. There is configuration option in voipmonitor.conf [[Sniffer_configuration#cleandatabase]]. |
If you do not have partitions you cannot use this option and you need to create some script which will delete old data from cdr table. This method is very slow since the delete has to update giant index file if tables are not partitioned. | If you do not have partitions you cannot use this option and you need to create some script which will delete old data from cdr table. This method is very slow since the delete has to update giant index file if tables are not partitioned. | ||
Latest revision as of 13:54, 3 February 2021
PCAP spool directory
By default sniffer stores all data to /var/spool/voipmonitor which can be changed in voipmonitor.conf - spooldir = ...
Version >= 8.0 implements new cleaning in voipmonitor.conf. Cleaning procedure runs every 5 minutes and checks the size or days according to following options. If you set maxpoolsize it will wipe out the oldest data every hour until the size is reached. maxpooldays keeps maximum number of data to set days. The same is for sip rtp and graph so you can keep sip pcaps longer than rtp pcaps. All following options can be activated at onc and all sizes are in MB
#set default maxpoolsize to 100 GB (102400 MB) maxpoolsize = 102400 maxpooldays = maxpoolsipsize = maxpoolsipdays = maxpoolrtpsize = maxpoolrtpdays = maxpoolgraphsize = maxpoolgraphdays = maxpoolaudiosize = maxpoolaudiodays =
Detailed description:
each created file is indexed in /var/spool/voipmonitor/filesindex/ in hours interval and the file size is added to aggregation mysql table files. Cleaning procedure iterates through index files and unlink files without need to scan directories.
If you accidentally remove /var/spool/voipmonitor/filesindex/ or files table in database, you need to reindex the spool directory otherwise the old not indexed files will be never deleted by the cleaning procedure. To initiate reindex send reindexfiles to manager interface
telnet localhost 5029 reindexfiles [enter]
Database cleaning
Since version 7 the sniffer is partitioning tables by days which allows efficient cleaning of partitions. There is configuration option in voipmonitor.conf Sniffer_configuration#cleandatabase.
If you do not have partitions you cannot use this option and you need to create some script which will delete old data from cdr table. This method is very slow since the delete has to update giant index file if tables are not partitioned.