DOC AI Cam: Manual for bird monitoring

The Bird Monitor on the DOC AI Cam provides an objective way of measuring the bird life population and how it’s changing. It does this by making one minute audio recordings at random times of the day and uploading these recordings, with GPS and time to the Cacophony servers for analysis.

A “Cacophony Index” is calculated for each recording - giving a measurement of the amount of bird song. It also automatically identifies which birds are in each recording. The hope is that the Cacophony Index will increase as you improve the birds’ environments through predator control and planting.

Even though we don't currently have a perfect way of analysing these audio files (we don’t recognise all birds) it is worthwhile to start recording now, so we can collect the data and analyse them when we have developed the technology. This technology is advancing rapidly and the types of analysis in the future are likely to be very sophisticated. The key is to start recording as soon as possible.

The operation of the thermal camera functionality of the DOC AI Cam is outlined in a different manual. This includes a description of how to connect to a camera, assign it to a group, update the location etc. This manual focuses on the Bird Monitoring function.

Note: We used to call the Bird Monitor a Cacophonometer. However, we have dropped this because it was very difficult to say and spell, and no one knew what it was.

The Bird Monitor was developed by The Cacophony Project. This not for profit organisation is developing technology to help eradicate predators from New Zealand, with devices for monitoring birds and predators and high interaction rate traps. 2040 LTD was formed to make New Zealand predator free 10 years before the government’s 2050 goal. We believe this can be achieved through the intelligent use of technology and our primary focus is to manufacture, sell and support the technology developed by The Cacophony Project. 2040 LTD helps fund The Cacophony Project by donating a proportion of all sales to them.

Privacy

The Bird Monitor may make recordings that include human conversations. By default if a human voice is detected then the whole recording will be deleted. This automatic deletion may be changed by the administrator of a group.

If you do record by mistake, you can delete the recordings via the website..

Note that recordings made by the Bird Monitor may be listened to by staff from The Cacophony Project (this includes approved, open source developers) and from 2040 Limited. The primary reason for this will be to develop algorithms to analyse the recordings automatically.

The user agreement for the Bird Monitor can be found here:
https://www.2040.co.nz/pages/2040-end-user-agreement

Audio modes

When recording audio the DOC AI Cam will make one minute recordings at random times through the day and night at a rate of 32 recordings per day.

The DOC AI Cam has 4 audio modes:

Video only, no audio
Audio only, no video
Video or audio (audio outside of the video recording window)
Video and audio

The first two modes should be self explanatory.

3.1 Video and Audio

In this mode audio recordings are made all day and night. However when a one minute audio recording is happening the camera is unable to make a video recording. So there is a chance you will miss a predator if it is in the field of view of a camera for only that minute.

If you are monitoring predator levels and have lots of predators, then this mode is recommended. It allows you to monitor predators with the camera and birds active during the night with the microphone.

3.2 Video or Audio

In this mode the DOC AI Cam will only record video or audio with video having the preference based on its recording window.

This mode is for when you are recording nocturnal predators with the video and you don’t want to miss any predators. For example you may be trying to hunt the last few predators in your area, or detecting re-invaders.

In this mode you will only get audio recordings in the daytime if you’re using the default video recording window (half an hour before dusk to half an hour after dawn). If you’re using 24 hour mode you won’t get any audio recordings.

In this mode the DOC AI Cam will make less than 32 recordings/day. This will be in proportion to how long the recording window is. For example if the recording window is for 12 hours it will average 16 recordings for the day. In the default mode the recording window varies as the length of the night changes. This changes throughout the year depending on the exact location of the device.

3.3 Selecting an audio mode.

You can select an audio mode by connecting the camera to your phone using the Sidekick App and selecting the mode on the Audio tab.

Manual recording

You can make a manual recording on the audio tab. This is useful to test the functionality or to make a record of a 5 minute bird count.

To make a manual recording select the audio tab and press the “10 Sec” button. Note: If there are human voices in the recording then this will be deleted as soon as the recording is processed (unless you have turned that option off). We recommend making non-vocal sounds for your test recordings.

If your DOC AI Cam is connected to the internet with WiFi then this recording should be uploaded immediately. If it has a modem the recording will be uploaded in about 10 minutes. If it’s not connected to the internet the recording will be stored on the camera and can be downloaded to Sidekick then uploaded to the Cacophony servers as described below.

To listen to the recording log on to the Cacophony Portal, click on recordings and you should be able to see your recording and listen to it.

5. Downloading recordings from your DOC AI Cam

You can download audio recordings from your DOC AI Cam in the same way you download thermal videos. This is described here.

You can see how many recordings it has stored by connecting the Sidekick app to your device. To upload the recordings from the DOC AI Cam you can either:

Connect it to WiFi (this could be your phone’s hotspot) and it will upload the recordings
Download the recordings to Sidekick and then upload them when your phone has an internet connection.

To download the recordings, connect your phone to Sidekick and press the download button

Deployment

Put the DOC AI Cam somewhere where the birds will be and make sure an audio mode is selected (as described above).

Reviewing recordings, automatic classifications, and tagging

To review your recordings navigate to https://browse.cacophony.org.nz/. Login and then click on your group name. Here’s a video demonstrating these features

You will then see a list of your recordings.

On this page you can:

Click the export button on the top right to export the data to CSV file that can be used in a spreadsheet
Change the view of the page to rows rather than cards that lets you see more recordings on a page.
Each recording has a card that includes:

The group name - clicking on this will take you to the group page
The device name - clicking on this will take you to the device page.
A graph of the Cacophony Index numbers for this recordings - you can see more detail about this here
A list of any tags that have been automatically or manually added to a recording. The colour of the tags is as follows:

Red: The tag was added automatically
Blue: The tag was added automatically and confirmed by a person
Green: The tag was added by a person:

The location icon - clicking on that will open a map showing where the recording was made (assuming the GPS was set correctly)
Clicking on anywhere on the tab will open up the page for listening to the recording and tagging.

Search options

The right hand side of the page contains the search options:

You can by device or group name.
You can select the recording type. this should be set to audio to access audio recordings. Note this same page is used for accessing videos made from thermal video cameras.
You can also select the date range.

There are additional search options under the advanced search, including:

The ability to select the duration of the recording. Normal recordings made by the bird monitor don’t vary - so most people won’t need this option.
Tag type. These tags are used for both audio and video recordings. The relevant tag types for audio recordings are the top 6 highlighted below.
tagged as - this lets you search, say for all the recordings that have been automatically tagged with morepork tags.

Listening to recordings

When you click on a recording card it opens up the recording page, where you can listen to the recording by pressing the play button. The recording is shown as a spectrogram at the top. This is a visual representation of the frequencies in the recording over time. This lets you easily see where interesting sounds might be in the recording and can make the process of identifying those interesting sounds much more efficient.

From the navigation bar you can navigate to the next or previous recording, download, or delete the recording. Below the navigation bar you can see the Cacophony Index (calculated for every 20s of recording). Hovering over this will show you the value of the Cacophony Index. Below the Cacophony Index is an interactive map showing the location of the recording. Below that is a link that will show you technical details of the recording and the state of the device when the recording was made.

Click on the if you would like to zoom into the recording. Click on the Zoom controls to change the level of Zoom. Click the to remove the zoom and see the original spectrogram.

Another type of zoom is being able to control which frequencies you can see. To change this, click the symbol and change the sample rate. Click the tick to complete.

You can also change the colours that are used in the spectrogram. To do this, click on the symbol and select the name of a colour scheme. This is useful for people that are colour blind and some colour schemes have better contrast for different recordings. This can be a matter of personal preference.

Reviewing tags

Any tags that have been added manually or automatically are shown underneath the spectrogram under the title classification. Each tag is associated with a tag box on the spectrogram. Clicking on the tag will play the audio recording in the tag box. You can confirm that the tag is correct by clicking the confirm button , or you can choose the correct label either by clicking on the label buttons, or selecting a label from the label drop down. There are a lot of birds in the label drop down - so we recommend using the search function - just type a few characters and you can find what you’re looking for.

The six buttons below the label box show the labels you select most often. You can also show force a label to show by using the pin button. The bottom 3 buttons don’t change.

Adding tags

To add a tag simply draw a box around the part of the spectrogram, that part of the recording will play and you can select a label from the drop down box. This data will then be used to build and improve automatic classification of the species you chose. Please only select a label if you are sure it’s right. If you know it’s a bird but are not sure what type then please just mark it as a bird. This is still useful. Please also mark other sounds such as wind, rain, vehicles, etc. It’s useful for us to be able to automatically identify these as well.

The Cacophony Index

The Cacophony Index is a measure of the amount of bird song. It's a number between 0 and 100 that is calculated for every 20 seconds of recording. So for our typical 1 minute recordings, you get 3 numbers.

The index is designed to be robust against many types of noise, such as wind, rain, aircraft engines and other common non-animal sounds.

It assumes the audio is from a recorder outdoors in a natural setting, away from non-natural sound sources, such as music, motorbikes or talking, any of which can produce very high Cacophony Index numbers in the 80s and 90s.

Some natural sounds, such as ocean waves or water dripping in a repeated pattern, will also confuse the index, so care must be taken if abnormally high cacophony index numbers are reported. We intend to improve this over time.

The Cacophony Index is shown as a graph on both the search page and the recordings page, with a value for each 20s of recording. If you hover over the graph it will show you the value of the Cacophony Index. You can also see the Cacophony Index if you export the recordings meta data to a spreadsheet.

From the spreadsheet here you can have a play with the data and see what you can find. This blog post contains some examples of the sort of things you can do.

It should be interesting to see:

how birdsong varies through the year.
If you're trapping, is your volume of birdsong going up
How your index compares to others?
If your index is going down then maybe you've got some evidence that you need to do some more control.

Species Classification

The Species Classification column in the export file shows which species have been automatically identified by our current algorithm. This is presented as a list of the species and the time in the recording where that species was identified. For example:

The Cacophony Project is regularly expanding the number of birds it recognises. This page contains an up to date list of the birds that are currently recognised.

One way you can help is to tag any birds that you do recognise in your own recordings. These tags will be used to train recognition algorithms. The more examples we have, the better these algorithms will be.

Software updates

There is a team of people actively working on the software. The DOC AI Cam will automatically check for updates any time it is connected. If you have a device that is not normally connected to the internet then it will be useful to connect it to the internet and update it if there are any significant updates.

We will write about any updates on the 2040 blog: https://www.2040.co.nz/blogs/news/tagged/bird-monitor. If you would like to be emailed about updates then please subscribe to our newsletter at the bottom of the 2040 homepage.

Frequently Asked Questions

What is the range of the bird monitor

The answer to this question is a little nuanced because it will depend on the volume of the bird, the weather (rain and wind will reduce the effective range to almost zero), foliage and terrain.

The microphone used is the same that is used in other bird recorders.

Can the bird monitor detect bats

The microphone is rated for the ultrasonic frequencies for bats, so we should be able to record bats eventually. At the moment this will require a software update to record at the higher frequencies.

How do I stop recordings with human voices being deleted automatically

By default recordings with human voices are deleted. This can be disabled by un-checking the Filter Audio Recordings of Human Speech check box which can be found at the top of your groups page.

If a human voice is detected we remove the recording, but keep all of the metadata so you can still see which sounds we automatically identified.

In the search results a recording that has been deleted will be shown in grey as below.

When you view a deleted recording you can see all the tags that were added automatically, but you cannot listen to the recording.

How long does the battery last?

This will depend on the exact configuration you have. If you’re using the bird monitor with the camera, it will have a negligible impact on the battery life. The camera uses significantly more power than the bird monitor.

If you’re using the camera in bird monitor only mode you may get as much as 200 days of monitoring using the recycled Lime battery. We haven’t done extensive testing on this yet because we haven’t had time. We will update this section as we get more data.

Acknowledgements

Many thanks to the team and contributors to The Cacophony Project for their work to create this product. Special thanks to Tim Hunt for donating his time and expertise to create the first version of the bird monitor

Visit https://cacophony.org.nz/ for the latest happenings.