Frigate NVR Updates
It's been a while since I did any videos or blog posts on the Frigate NVR. Someone mentioned to me some new functionality that might be in version 13 (which is in beta as of this writing) so I thought I would check it out.
It's been a while since I did any videos or blog posts on the Frigate NVR. It has been running smoothly and I haven't had the need to do anything so I just left it to run. Someone mentioned to me some new functionality that might be in version 13 (which is in beta as of this writing) so I thought I would check it out.
It turns out there are some significant changes from the current v12.x to the new v13, including breaking changes. I'll get into those in a bit. First, let me summarize the new features and changes.
I run Frigate in a Docker container on a Dell Optiplex. You can watch my video on how I installed it.
Frigate+
If you have been paying attention, in addition to Frigate standalone, there is Frigate+, a subscription-based service that provides more specific models. From their website:
Frigate+ offers models trained from scratch and specifically designed for the way Frigate NVR analyzes video footage. These models offer higher accuracy with less resources. By uploading your own labeled examples, your model is tuned for accuracy in your specific conditions. After tuning, performance is evaluated against a broad dataset and real world examples submitted by other Frigate+ users to prevent overfitting.
Frigate+ models are also trained to detect a more relevant set of objects for security cameras. Currently, the following objects are supported: person, face, car, license plate, amazon, ups, fedex, package, dog, cat, deer. Additional object types will be added in future releases.
With the release of Frigate v13, Frigate+ models are rolling out by invitation. You will need a Frigate+ subscription for this. The models can be automatically downloaded from Frigate+. In addition, you can submit false positives to Frigate+ for training the models. All of this information is available on their GitHub releases page.
Support for SBC and other devices
Version 0.12 of Frigate included support for Intel iGPUs and Nvidia GPUs. With v13, a community-supported board framework is rolling out to allow the community to add support for specific boards. For example, Jetson 4.6 and Jetson 5 devices are now supported as well as many RockChip SoCs.
Object Tracking/Motion Detection
One of Frigate's main features is the ability to detect objects so it is only fitting that there are updates to that area. This allows Frigate to be more efficient and works to reduce false positives.
- In v13, Frigate switched to using Norfair for object tracking. With that comes many improvements in how objects are identified and tracked.
- Motion detection is more efficient and will re-calibrate when big lighting changes occur (switch to IR mode, lightning, PTZ movements, etc.).
- The recording timeline is used to improve detection accuracy after motion and reduces false positives by creating an 8x8 grid of expected regions for the camera, which is updated daily.
- It is now possible to set the minimum amount of time an object needs to be in a zone before it is considered to actually be in that zone. This inertia setting is particularly useful if someone briefly passes through a zone.
Recordings
This is one of my favorite improvements!
- It is now possible to export recordings either as a regular speed video or as a time-lapse based on user-selectable time/date ranges. You can download and manage those exports directly from the UI. Getting a timelapse of a specific time was always problematic and required that I stitch together tens or hundreds of small video files into one big file. The export feature solves this.
- Timeline metadata is now stored for specific moments during an event. It is now possible to highlight when an object is detected, enters or leaves a zone, or becomes stationary. This is done by simply clicking icons above a recording. This makes it very easy to tell what triggered an event and what an object was doing during that event. There is a new setting called 'annotation_offset' that allows adjustment of the overlay to match the actual object detection.
- Lots of efficiency improvements
- Recordings maintainer now lives in a dedicated process
- Recordings maintenance is now fully asynchronous
- Database writes are now much more efficient
- Record config can now have
sync_on_startupenabled which will check for recordings in the db that are not on the disk and delete them
Audio Triggering!
Audio event triggering is now supported using YamNet. There are over 600 different sounds that Frigate can listen for. If you are using Home Assistant integration, the 5.0 version of the integration supports sound sensors which means you can create automations based on what sound was heard. Amazing!
Manual Events via API
It is now possible to create events manually via the API. You can use this for when a sensor is tripped or someone presses the doorbell, etc. You can set the event length, recording save preferences, and more. There is full documentation here.
PTZ
Frigate can now support PTZ cameras with auto-tracking. This is something that I have struggled with in the past. I want to be able to monitor what my dog is doing in the house when I am gone but the field of view for my cheapo PTZ camera doesn't have a wide enough field of vision to cover the area I am interested in. This will be fun to try out.
Breaking Changes
As with any major release, there are bound to be some breaking changes. That is the case with the v13 release. Here are the highlights.
- Since the detection model has changed, so have the default values for motion detection. The recommendation is that you remove any specific values in your config and start tuning/tweaking from scratch.
- If you have configured settings for stationary objects and have set the interval to zero, you must either delete that setting or increase the value. This is because stationary object validation is required.
- SUPER IMPORTANT. Changes to the database have been made that will prevent you from reverting to a previous version. Make sure you take a backup of your database BEFORE upgrading to v13 as the DB will be updated with the new schema.
- The default location for the database has been moved. It is now in /config/frigate.db. The move to the new location is automatic and may require a config change depending on how you run Frigate (add-on, Docker). For the add-on, nothing is needed. For Docker users, you will need to change your volume mapping to /host/path/config_folder/:/config/
- Configuration for retain days needs to follow the record -> retain -> days format. Some of you may have already changed this in your config. I did.
- As mentioned earlier, the Home Assistant 5.0 integration is required for all features to work, especially sound alerts. I was able to upgrade without updating to the beta integration but I didn't do anything with sound.
- If you are using TensorRT, it now requires the Nvidia driver version 530 or greater.
My Config
Now that I have gone through all the updates/changes/etc, here is what my config file looks like while running v13. As of this writing, I am on 0.13.0 Beta5. I had to remove some of the stationary settings as mentioned above, as well as reset my motion detection preferences to tune for the new model being used.
ui:
use_experimental: false
live_mode: mse
mqtt:
host: 172.31.10.3
port: 1883
topic_prefix: frigate
client_id: frigate
user: mqttuser
password: mqttuser
stats_interval: 300
record:
expire_interval: 10
detect:
annotation_offset: -800
go2rtc:
streams:
driveway:
- rtsp://user:password@172.31.10.20:554/cam/realmonitor?channel=1&subtype=0
driveway_sub:
- rtsp://user:password@172.31.10.20:554/cam/realmonitor?channel=1&subtype=1
front_porch:
- rtsp://user:password@172.31.10.21:554/cam/realmonitor?channel=1&subtype=0
front_porch_sub:
- rtsp://user:password@172.31.10.21:554/cam/realmonitor?channel=1&subtype=1
east_side_cam:
- rtsp://user:password@172.31.10.22:554/cam/realmonitor?channel=1&subtype=0
east_side_sub:
- rtsp://user:password@172.31.10.22:554/cam/realmonitor?channel=1&subtype=1
west_side_cam:
- rtsp://user:password@172.31.10.24:554/cam/realmonitor?channel=1&subtype=0
west_side_sub:
- rtsp://user:password@172.31.10.24:554/cam/realmonitor?channel=1&subtype=1
garage:
- rtsp://user:password@172.31.10.152:554/h264Preview_01_main
garage_sub:
- rtsp://user:password@172.31.10.152:554/h264Preview_01_sub
back_porch:
- rtsp://user:password@172.31.10.23:554/cam/realmonitor?channel=1&subtype=0
back_porch_sub:
- rtsp://user:password@172.31.10.23:554/cam/realmonitor?channel=1&subtype=1
front_doorbell:
- rtsp://userpassword@172.31.10.128:554/cam/realmonitor?channel=1&subtype=0#backchannel=0
front_doorbell_sub:
- rtsp://userpassword@172.31.10.128:554/cam/realmonitor?channel=1&subtype=1#backchannel=0
timestamp_style:
position: tl
format: '%m/%d/%Y %H:%M:%S'
color:
red: 255
green: 255
blue: 255
thickness: 2
effect: shadow
cameras:
driveway:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/driveway_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/driveway
input_args: preset-rtsp-restream
roles:
- record
output_args:
record: preset-record-generic-audio-aac
detect:
width: 640
height: 480
fps: 5
objects:
track:
- person
- dog
- bicycle
- cat
- car
snapshots:
enabled: true
timestamp: true
bounding_box: true
required_zones:
- driveway_left_side
- driveway_right_side
- front_yard
- front_street
crop: true
height: 500
retain:
default: 3
zones:
driveway_right_side:
coordinates: 505,131,604,128,611,459,466,459,361,480,188,480,298,284,179,261,242,210,320,138
#inertia: 1
objects:
- person
- dog
- cat
- bicycle
driveway_left_side:
coordinates: 272,305,194,426,143,480,43,394,109,323,179,257
#inertia: 1
objects:
- person
- dog
- cat
- bicycle
- car
front_yard:
coordinates: 145,278,299,148,152,165,29,256
objects:
- person
front_street:
coordinates: 80,183,187,155,337,132,640,123,640,0,502,0,289,50,0,161,0,251
objects:
- person
- bicycle
motion:
mask:
- 640,0,640,41,451,40,450,0
record:
enabled: true
retain:
days: 3
events:
retain:
default: 3
mode: active_objects
required_zones:
- driveway_left_side
- driveway_right_side
- front_yard
- front_street
pre_capture: 3
post_capture: 10
front_porch:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/front_porch_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/front_porch
input_args: preset-rtsp-restream
roles:
- record
output_args:
record: preset-record-generic-audio-aac
detect:
width: 640
height: 480
fps: 5
objects:
track:
- person
- dog
- cat
snapshots:
enabled: true
timestamp: false
bounding_box: false
crop: true
height: 500
required_zones:
- front_porch_close_in
retain:
default: 3
motion:
mask:
- 441,42,640,43,640,0,444,0
zones:
front_porch_close_in:
coordinates: 640,0,640,480,201,480,156,301,112,0
record:
enabled: true
retain:
days: 0
events:
retain:
default: 3
mode: active_objects
required_zones:
- front_porch_close_in
pre_capture: 3
post_capture: 10
front_doorbell:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/front_doorbell_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/front_doorbell
input_args: preset-rtsp-restream
roles:
- record
output_args:
record: preset-record-generic-audio-aac
detect:
width: 720
height: 576
fps: 15
objects:
track:
- person
- dog
- bicycle
- cat
- mouse
snapshots:
enabled: true
timestamp: false
bounding_box: false
crop: true
height: 500
retain:
default: 3
motion:
mask:
- 720,0,720,28,430,22,428,0
record:
enabled: true
retain:
days: 0
mode: active_objects
events:
retain:
default: 3
mode: active_objects
pre_capture: 10
post_capture: 15
east_side_cam:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/east_side_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/east_side_cam
input_args: preset-rtsp-restream
roles:
- record
output_args:
record: preset-record-generic-audio-aac
detect:
width: 640
height: 480
fps: 5
stationary:
max_frames:
objects:
person: 1000
objects:
track:
- person
- dog
- mouse
- cat
filters:
person:
mask:
- 434,480,640,480,640,0,450,0
snapshots:
enabled: true
timestamp: true
bounding_box: true
crop: true
height: 500
retain:
default: 3
zones:
side_of_house:
coordinates: 472,480,0,480,100,279,351,54,456,42
motion:
mask:
- 464,232,424,480,640,480,640,0,439,0
record:
enabled: true
retain:
days: 0
events:
retain:
default: 3
mode: motion
pre_capture: 3
post_capture: 10
back_porch:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/back_porch_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/back_porch
input_args: preset-rtsp-restream
roles:
- record
detect:
width: 640
height: 480
fps: 6
objects:
track:
- person
- dog
- cat
- mouse
filters:
person:
mask:
- 192,480,0,480,0,94
- 538,331,366,480,640,480,640,90
snapshots:
enabled: true
timestamp: false
bounding_box: true
retain:
default: 3
motion:
mask:
- 199,480,171,433,0,110,0,480,40,480
- 640,163,640,480,536,480,387,480
zones:
deck_area:
intertia: 2
coordinates: 105,0,436,0,484,21,640,108,640,290,640,398,427,480,181,480,108,352,62,257,0,112,0,39
record:
enabled: true
retain:
days: 0
events:
retain:
default: 3
mode: motion
pre_capture: 5
post_capture: 15
west_side_cam:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/west_side_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/west_side_cam
input_args: preset-rtsp-restream
roles:
- record
output_args:
record: preset-record-generic-audio-aac
detect:
width: 640
height: 480
fps: 5
objects:
filters:
person:
mask:
- 314,480,0,480,0,0,199,0
snapshots:
enabled: true
timestamp: true
bounding_box: true
crop: true
height: 500
retain:
default: 3
zones:
west_side_of_house:
coordinates: 323,78,478,65,640,174,640,290,640,480,310,480,263,321,193,0,268,0
motion:
mask:
- 442,44,640,44,640,0,439,0
- 0,480,319,480,233,204,196,0,0,0
record:
enabled: true
retain:
days: 0
events:
retain:
default: 3
mode: motion
pre_capture: 3
post_capture: 10
#
garage:
ffmpeg:
inputs:
- path: rtsp://127.0.0.1:8554/garage_sub
input_args: preset-rtsp-restream
roles:
- detect
- path: rtsp://127.0.0.1:8554/garage
input_args: preset-rtsp-restream
roles:
- record
output_args:
record: preset-record-generic-audio-aac
detect:
width: 640
height: 480
fps: 5
objects:
track:
- person
- dog
- cat
- car
filters:
person:
mask:
- 208,92,619,91,640,0,218,0
zones:
garage_entrance:
coordinates: 299,307,163,346,343,436,487,358
snapshots:
enabled: true
timestamp: false
bounding_box: true
retain:
default: 3
record:
enabled: true
retain:
days: 0
events:
retain:
default: 2
mode: active_objects
pre_capture: 5
post_capture: 15
motion:
mask:
- 208,98,532,96,539,0,209,0
detectors:
coral:
type: edgetpu
device: usb
rtmp:
enabled: false
birdseye:
enabled: true
restream: true
width: 1280
height: 720
quality: 1
mode: objects
Alerts in Home Assistant
I find one of the most useful features of Frigate is alerting when an object is in one of my zones of interest. I use Home Assistant with the Frigate integration for this. I have automations that alert me when a person is in a zone and I also use automation to turn on a distance transducer when a car is detected entering the garage.
With the latest versions of Home Assistant, many of the pieces of this automation can be done via the UI. However, I will post the YAML code below so you can import it into an automation and adjust it as needed.
I based my automation on a non-blueprint automation in this post, which explains much of what is happening. There are blueprints that you can import for Frigate notifications, but I needed better control of how the automation runs.
Alert when tracked object enters a zone:
alias: Frigate Driveway w Image
description: ""
trigger:
- platform: mqtt
topic: frigate/events
payload: driveway
value_template: "{{ value_json['after']['camera'] }}"
id: frigate-event
variables:
after_zones: "{{ trigger.payload_json['after']['entered_zones'] }}"
before_zones: "{{ trigger.payload_json['before']['entered_zones'] }}"
camera: "{{ trigger.payload_json['after']['camera'] }}"
id: "{{ trigger.payload_json['after']['id'] }}"
label: "{{ trigger.payload_json['after']['label'] }}"
score: "{{ trigger.payload_json['after']['score'] }}"
time_clip_start: "{{ trigger.payload_json['after']['start_time'] - 10.0 }}"
condition:
- condition: template
value_template: >-
{{ not this.attributes.last_triggered or (now() -
this.attributes.last_triggered).seconds > cooldown }}
alias: Wait for cooldown
- condition: or
conditions:
- condition: template
value_template: "{{ trigger.payload_json['type'] == 'new' }}"
- condition: template
value_template: "{{ trigger.payload_json['before']['entered_zones'] | length == 0 }}"
- condition: template
value_template: "{{ trigger.payload_json['after']['entered_zones']|length > 0 }}"
- condition: template
value_template: >-
{{ ["driveway_left_side", "driveway_right_side"] | select("in",
after_zones) | list | length > 0 }}
- condition: template
value_template: "{{ trigger.payload_json['after']['label'] != 'car' }}"
action:
- choose:
- alias: Frigate Event
conditions:
- condition: trigger
id: frigate-event
sequence:
- variables:
id: "{{ trigger.payload_json['after']['id'] }}"
- alias: Notify on new object
choose:
- conditions: []
sequence:
- service: notify.mobile_app_sm_s906u1
data:
message: Driveway Motion
title: Driveway Alert
data:
image: "{{base_url}}/{{id}}/thumbnail.jpg?format=android"
clickAction: "{{action_url}}/api/events/{{id}}/clip.mp4"
actions:
- action: URI
title: View Clip
uri: "{{action_url}}/api/events/{{id}}/clip.mp4"
channel: drive_Cam
priority: high
ttl: 0
- service: notify.hass_agent_superman
data:
message: Driveway Motion
title: Driveway Alert
data:
image: "{{base_url}}/{{id}}/thumbnail.jpg"
mode: parallel
variables:
base_url: >-
https://{mynabucasa_URL}.ui.nabu.casa/api/frigate/notifications
action_url: https://{remotely_accessible_url_to_my_frigate_instance}.com
cooldown: 90
Car enters garage and activates atom matrix light/transducer:
alias: Frigate Car Enter Garage
description: ""
trigger:
- platform: mqtt
topic: frigate/events
payload: garage
value_template: "{{ value_json['after']['camera'] }}"
id: frigate-event
variables:
after_zones: "{{ trigger.payload_json['after']['entered_zones'] }}"
before_zones: "{{ trigger.payload_json['before']['entered_zones'] }}"
camera: "{{ trigger.payload_json['after']['camera'] }}"
id: "{{ trigger.payload_json['after']['id'] }}"
label: "{{ trigger.payload_json['after']['label'] }}"
score: "{{ trigger.payload_json['after']['score'] }}"
time_clip_start: "{{ trigger.payload_json['after']['start_time'] - 10.0 }}"
condition:
- condition: template
value_template: >-
{{ not this.attributes.last_triggered or (now() -
this.attributes.last_triggered).seconds > cooldown }}
alias: Wait for cooldown
- condition: or
conditions:
- condition: template
value_template: "{{ trigger.payload_json['type'] == 'new' }}"
- condition: template
value_template: "{{ trigger.payload_json['before']['entered_zones'] | length == 0 }}"
- condition: template
value_template: "{{ trigger.payload_json['after']['entered_zones']|length > 0 }}"
- condition: template
value_template: "{{ [\"garage_entrance\"] | select(\"in\", after_zones) | list | length > 0 }}"
- condition: template
value_template: "{{ trigger.payload_json['after']['label'] == 'car' }}"
action:
- choose:
- alias: Frigate Event
conditions:
- condition: trigger
id: frigate-event
sequence:
- variables:
id: "{{ trigger.payload_json['after']['id'] }}"
- alias: Notify on new object
choose:
- conditions: []
sequence:
- service: notify.mobile_app_sm_s906u1
data:
message: Car in garage
title: Car in Garage
data:
image: "{{base_url}}/{{id}}/thumbnail.jpg?format=android"
channel: car_in_drive
priority: high
ttl: 0
- service: notify.hass_agent_superman
data:
message: Car in garage
title: Car in Garage
data:
image: "{{base_url}}/{{id}}/thumbnail.jpg"
clickAction: "{{action_url}}/api/events/{{id}}/clip.mp4"
actions:
- action: URI
title: View Clip
uri: "{{action_url}}/api/events/{{id}}/clip.mp4"
- service: switch.turn_on
data: {}
target:
entity_id: switch.garage_atom_matrix
- delay:
hours: 0
minutes: 2
seconds: 0
milliseconds: 0
- service: switch.turn_off
data: {}
target:
entity_id: switch.garage_atom_matrix
mode: parallel
variables:
base_url: >-
https://{mynabucasa_URL}.ui.nabu.casa/api/frigate/notifications
action_url: https://{remotely_accessible_url_to_my_frigate_instance}.com
cooldown: 90
That's a quick (or long) rundown of the new stuff in Frigate 13 as well as how I have configured my setup and the automations that I use for notifications and other things based on detection. Don't forget to watch my related video on this topic.
Until next time...