Dummies Guide

From ZoneMinder Wiki
Revision as of 15:31, 12 February 2017 by Snake (talk | contribs) (→‎Notes)
Jump to navigationJump to search

Notes I've recorded as I've learned ZoneMinder.

Install:

Use the installation guides provided on the: Zoneminder Wiki: Contents

I've used trisquel and devuan.

Some installation guides omit the required dependencies for libvlc to work. Search forum if you need libvlc.

Here's a guide for using an external HDD: Using a dedicated Hard Drive

Make sure you add innodb_file_per_table=1 and ensure that the ZM database is InnoDB format, otherwise you may run into the issues described in the mysql section.

Test out a Camera:

Once you get ZM installed, you will want to test out a camera. You can do a webcam, or you can do an IP camera. See the Hardware Compatibility LIst

An Axis is a good start. They are well supported (though HD is expensive). Buy an old one for $10-20.


Follow the instructions in the Hardware Compatiblity List for parameters for setting up a camera the first time. If you have an error, look at the logs.

Resolution must be correct. Many other fields can be left at defaults.

If you can view the MJPEG or JPG url in your browser, you will be in good shape.

Obtaining more Cameras:

In ZoneMinder, when you add a camera, you have a few options.

  • LOCAL Camera connected directly to computer (webcam, or analog camera thorugh bttv card)
  • REMOTE Grab mjpeg stream or rtsp (h264) stream from IP camera on the network
  • FILE Grab a jpeg file somewhere on the server and display that (you provide jpeg images that change from somewhere)
  • FFMPEG and LIBVLC use the respective libraries to pull a stream similar to REMOTE does for RTSP only.

Analog cameras are limited in resolution (about 700x400px). So called HD Analog cameras are not compatible with ZM. However analog cameras can use a network encoder to turn an analog stream into an IP Camera RTSP stream. After that you have FILE and REMOTE. I have not used FILE for anything but one off cameras. I will cover REMOTE.


REMOTE gives you the option of either RTSP (h264) or Mjpeg cameras. Both will work.


Mjpeg

These include Arecont Vision, Axis, Bosch, Foscam, Grandstream, Instar, Messoa, Zavio and others. The prices scale with features. Old Axis cameras at 480p resolution (no IR) can be found online easily for $10-50. New Foscams can be purchased at their online store for $80-130 with 1080p resolution (IR). Above that is beyond my budget. More pixels means higher cost.

Also see, SBC cameras.

RTSP

These cameras use h264 compression. However ZM decodes all incoming video to jpeg files, so it's not optimized. This requires libvlc, or ffmpeg to convert the video to jpegs for zoneminder. Same as MJPEG, the prices go up as you get more resolution. However it's easier to find a 1080p RTSP camera, for $70 vs. an 1080p MJPEG camera, generally.

Note: Exception is feature-h264-videostorage branch. This new branch takes h264 and writes it direct to an .mp4 container. Keep an eye on this branch.

Conclusion

Based on your needs you must choose either MJPEG, RTSP, or a mix. If you want a high quantity of cameras, it's more efficient to use all MJPEG cameras. If you have a small number, or If you want to purchase high-end server hardware, then it doesn't matter. With powerful enough server hardware, you will be able to run 10-20 HD cameras of either type without difficulty. In fact, a lot of my CPU usage comes from compressing the video (optional, see below).

Let me state again, high-end server hardware will perform better than desktop, or low end server hardware. I have seen this firsthand between two servers: KFSN4-DRE and the KGPE-D16. The latter runs ZM with 16 cameras, not breaking a sweat. The former reaches a limit at 10.

In my case, I needed the most affordable HD MJPEG cameras, so I searched and settled on the Foscam models. I'd also recommend used Arecont Vision cameras or Axis, if you can afford them. Beware that newer foscams do not have MJPEG streaming, and some firmware have issues with the stream not working. Read carefully on the foscam forums before purchasing. Or (better) use an SBC.

Watching the Cameras:

Cameras can be watched from ZM web server. Montage view is good. See notes below for > 6 camera setups.

Another way is to make an html file on a remote machine with the following code embedded in an img tag. Adjust monitor ID as needed. How to stream from another ZoneMinder installation. An easy way to embed a webcam in a website.

Streaming in ZM, does not slowdown it much, in my experience.

Recording in Zoneminder:

There are two approaches I use, with recording in Zoneminder. Record everything, and export old videos to a compressed format for safekeeping, or use motion detection and keep only what the HDD can store.

Though if desired, you can use both.

Method 1: Recording All & Exporting Videos Hack

If you set cameras to Record and if you have enough high resolution cameras you will see your disk space fill up fast. ZM records in 10 minute segments and captures every frame as a jpeg. There is no compression of similar jpegs. Also, higher frame rate means more jpegs, more disk space.

How do you avoid filling up the HDD with lots of cameras?

You hack ZM. Use a filter which will take recordings from a specified range and create a video for all matches. Then you make a filter that will delete the events which have already had videos created.

Or alternatively, you use motion detection (see below), recording only when motion occurs (modect, not mocord).

What do you mean filter?

On the zm home console, there is a filter button. Click that, specify the range of videos you want to create videos for, click the checkbox for 'create video for al matches', submit/save, select it to run in the background, done.

How is it going to make the video?

You can specify parameters in FFMPEG_OUTPUT_OPTIONS in the options of zm. Make sure you are using ffmpeg, not avconv. You also need to specify the path for ffmpeg. I used the ffmpeg binary.



I settled on the following path for ffmpeg output options:

  • -c:v libx264 -preset ultrafast /videosfolder/on/some/path/`date -d "-1 days" +"%Y%m%d-%H%M%S-%N"`.avi < /dev/null

VP9 compresses best, but is slower to encode. If you have hardware optimized for VP8 or VP9, maybe it will be better. Webm / MKV is great.

Here are a couple others I found on the forums, before settling on libx264: 

* ''-c:v libx264 -preset fast -crf 38 -vf hqdn3d=8:6:12

* -r 1 -f image2 -i %07d.jpg -vcodec libvpx-vp9 -crf 18 

* -r 25 -q:v 1  /mnt/where/my/videos/go/`date -d "-1 days" +"%Y%m%d-%H%M%S-%N"`.webm

* -r 1 -f image2 -i %07d.jpg -vcodec libx264 -crf 18 -preset slower''

More are on Zoneminder forums. The general idea is you can specify a codec with

-c:v libx264 OR -c:v libvpx-vp9

Find any codec you like, and put it in -c:v and it should encode. Find one your hardware supports well, and go from there.

So I used in my FFMPEG_OUTPUT_OPTIONS

-c:v libx264 -preset ultrafast /videosfolder/on/some/path/`date -d "-1 days" +"%Y%m%d-%H%M%S-%N"`.avi < /dev/null


This works, but we have a problem. All videos are in one folder. The next step was to make script Script for Deleting Old Exported Video Folders that would take files daily from this folder and put them into subdirectories labeled by the date. Then I made script Script for Deleting Old Exported Video Folders pt.2 that would measure the filesystem capacity of this folder (it is a dedicated drive) and if the capacity is over a set amount, delete the oldest two days.

This worked well, except that finding videos among the dozens in one day was slow.


The next step was based off of this post:

Event Video Saved to Custom Location by Default

So I edited zmvideo.pl and was able to get most of the cameras to export to their own folders (see above forum posts). Though now I had to change ffmpeg output options to:

-c:v libx264 -preset ultrafast

as now the perl script is managing where the output folders go.

I had to debug when setting up the perl script in order to make it work. Permissions, and watching the /var/log/zm/{zmvideo,zmfilter}.log helped.

All these steps for "Recording / Exporting Videos" are optional. It is recommended you use Motion Detection or Mocord, for most setups.

Method 2: Motion Detection

This is an option most people should use. Use a dedicated HDD, record on motion only, and you will retain sufficient history.


What about motion detection

The strength of zoneminder lies in its motion detection (thus the 'zone' in zoneminder, being the motion detection zones). See: Understanding Zoneminder's Zoning system for Dummies

Start with the most sensitive settings, and make them less sensitive until no false alarms are recorded. Monitoring for all alarms can easily be done by $ tail -f /var/log/syslog in Devuan based distros. Or you can record everything, if modect isn't working. Mocord will combine both modect and record, having everything recorded, but making a graph where motion occurs (in show timeline). Mocord is the best, but uses more HDD.


Is motion detection required?

Using modect is recommended here. Because you are able to click from the main zm index - events - show timeline, and from there quickly review your modect events. Because you save HDD space. And, SMS notifications can be setup off of modect alarms.

Troubleshooting:

  • Watch logs.
  • Use forum search.
  • Use web search.
  • If you click, 'component logs' in the options - logs section of zm, you will get log files in the /var/log/zm/ folder.
  • # tail -F /var/log/syslog
  • Beware of underlying hardware faults such as bad RAM.

Notes

  • Some cams will have two video streams (e.g. Hikvision). The resolutions/video type may or may not be the same. For example, there may be a low resolution mjpeg stream, and a high resolution RTSP stream.
  • I found it helpful to name the cameras after the monitor ID as you run into monitor ID in logs, often.
  • Proprietary cameras are known to report to outside IPs. Don't give them internet access. Only the server should be wan-accessible. Put all cameras on a physically separate network, or use VLANs.
  • Many cameras have default telnet passwords, in addition to the default web access passwords. Change these or keep cameras away from the wan. Cameras are common botnet targets.
  • With server motherboard hardware, you will be able to have more cameras (servers are more powerful, and better servers will have better performance).
  • I use ext4 filesystem for the HDDs. I had tried using ext2 filesystem for better performance, but the fsck time is prohibatively slow for ext2 (>24 hours for >2TB), therefore I do not recommend using it. Ext4 seems to work well. Older ext2, or ext3 fs can be upgraded to ext4.
  • If you are going to use a BT878 based card, you probably should add a heatsink onto the top of all *878 chips. This has been mentioned in the forums in various spots, and I noticed one of my chips out of the 3 on the board not working correctly after the first test. A heatsink with similar dimensions can be purchased from electronic stores such as digikey, mouser, adafruit, or sparkfun. Use a caliper or a ruler, and measure the dimensions in mm, then search for a similar heatsink. You will also need some kind of adhesive. I used an epoxy based thermal adhesive. A heatsink attached to the plastic of the IC, instead of the actual IC die is not ideal, but it will help.


Guides

  • Zmodopipe Is a tool that can tie an analog DVR system to Zoneminder, although it is far from perfect. I have documented it there, and recommend purchasing an analog network converter instead.
  • PurgeWhenFull requires configuration on larger systems, or systems where events are created at a pace faster than PurgeWhenFull can keep up.
  • ZMTrigger is a tool that can be used to take outside information and overlay it onto the camera display. For example, you might take the temperature, or wind speed, and overlay it on a camera. It also has other uses. Experience with microcontrollers such as AVRs, Pics, and the Arduino IDE are useful here.

MYSQL

IBData files Large

I had an issue with the ibdata1 file in /var/lib/mysql/ growing too large. It includes some database information and in my 10GB root partition, was taking up 8GB.

The solutions I found were:

OR

  • Move the ibdata file to another partition

OR

  • Change DB type to InnoDB (requires backup, deletion, and restoring db, per first solution)

Changing the database type to have an innodb file per each table as mentioned in the "how to shrink purge ibdata1 file in mysql" link will keep less data used in the ibdata1 file in the future, allowing the former to be deleted when not needed. On the other hand the ibdata file by default, will not shrink, ever. This may not be an issue in MariaDB.

Looking for the least invasive procedure, I went with moving /var/lib/mysql, and adding the optional my.cnf parameter. This required the following tricks (may only apply to Ubuntu 14.04).

There are a number of guides on moving Mysql, yet many of them omit adding the alias to apparmors settings. This is required. Failing to do so will result in "Job failed to start" when mysql is run with #service mysql start.

A guide that covers all the steps required to move mysql on Ubuntu Trusty without omitting anything is here: Ask Ubuntu: Moving Mysql datadir Note that within my mysql installation there was no socket file in /var/lib or in my.cnf.

Backup/Restore Mysql DB

After moving the Data directory, I ended up backing up the zm db and restoring it anyways, in order to get the ibdata files to split correctly. This is not hard to do. The only DB you need to mysqldump from a stock ZM installation is the ZM db. And it's also the only DB you need restore.

For a full walkthrough on converting a MyISAM DB to InnoDB (also covers backing up ZM DB) see Enable and convert MySQL to innodb file per table for Zoneminder.

SMS Notifications

For SMS notifications, setup run states, and define filters which send emails to an SMS gateway which will forward to your mobile. Use different run states, for turning the SMS notifications off and on (NOTE: this feature is currently in the process of being implemented. Until then you will have to have duplicate monitors, one for each runlevel, with one monitor having the SMS assigned to it.). See the following:

Retrieved from "https://wiki.zoneminder.com/index.php?title=Dummies_Guide&oldid=14356"