Birds recognised expanded to over 60
The DOC AI Cam includes a microphone and can optionally be configured to make audio recordings.
We have just released our latest bird recognition model that greatly expands the number of birds that we recognise to over 60. More accurately, it's over 60 sounds because this includes insect, frog, weta, human and noise.
This expanded list has been trained using tagged bird recordings supplied by DOC. Thank you very much to them for sharing this with us. When we tested the model against our test set it did a pretty good job at identifying the birds accurately, or if it was not able to do that it would identify them as "don't know". This should reduce the number of misclassifications.
Our algorithm is still using the data from eBird to restrict, by region which birds will be identified. Thank you to them too for letting us use this data.
The real test is how it goes in the wild (literally). Please let us know how well the new algorithm is doing.
Here's the complete list. This shows how the model performs against our test set. You can see that while the accuracy isn't that good at 42%, it will still label it as bird 31% which is accurate, but not specific. 6% of the time it will say unsure. It get's it wrong about 21% of the time. You can see in the table below what the most common mis-classified bird is. You can also see the number of samples we have for each bird in our test set (note this is different from the samples we use for training). These numbers should improve as we get more accuracy.
| label | correct % | misclassified label | misclassified % | unsure % | bird % | total samples |
| Average | 42.3% | 12.9% | 6.1% | 30.8% | ||
| bellbird | 19.9% | noise | 9.3% | 7.7% | 53.5% | 2014 |
| black noddy | 87.8% | white tern | 2.0% | 2.0% | 8.2% | 49 |
| black-swan | 0.0% | human | 50.0% | 50.0% | 0.0% | 2 |
| blackbird | 24.9% | noise | 11.3% | 8.8% | 43.3% | 799 |
| brown-creeper | 32.6% | noise | 6.5% | 6.5% | 43.5% | 46 |
| california quail | 0.0% | fantail | 20.0% | 20.0% | 20.0% | 5 |
| chaffinch | 33.0% | noise | 11.3% | 5.2% | 38.7% | 442 |
| crimson rosella | 37.1% | white tern | 4.8% | 0.0% | 50.0% | 62 |
| duck | 25.0% | noise | 25.0% | 0.0% | 50.0% | 4 |
| dunnock | 25.9% | chaffinch | 7.4% | 14.8% | 25.9% | 27 |
| fantail | 45.7% | noise | 9.9% | 4.9% | 31.9% | 698 |
| frog | 0.0% | noise | 94.1% | 5.9% | 0.0% | 17 |
| goose | 28.6% | noise | 28.6% | 28.6% | 14.3% | 7 |
| greenfinch | 11.1% | blackbird | 5.6% | 2.8% | 55.6% | 36 |
| grey warbler | 46.9% | noise | 10.3% | 6.7% | 30.3% | 1022 |
| indian-myna | 0.0% | None | 0.0% | 20.0% | 80.0% | 5 |
| insect | 81.6% | noise | 13.1% | 2.2% | 1.5% | 267 |
| kea | 60.6% | noise | 7.9% | 4.7% | 17.3% | 127 |
| kereru | 0.0% | noise | 28.6% | 0.0% | 42.9% | 7 |
| kiwi | 42.7% | noise | 18.5% | 5.6% | 7.9% | 178 |
| long-tailed-cuckoo | 61.1% | noise | 18.7% | 3.3% | 10.7% | 337 |
| magpie | 33.3% | morepork | 16.7% | 8.3% | 16.7% | 12 |
| marsh-crake | 0.0% | noise | 100.0% | 0.0% | 0.0% | 2 |
| mohua | 20.0% | noise | 40.0% | 0.0% | 0.0% | 5 |
| morepork | 65.2% | noise | 21.2% | 8.6% | 4.3% | 2722 |
| new-zealand-falcon | 8.3% | chaffinch | 8.3% | 33.3% | 25.0% | 12 |
| new-zealand-fernbird | 0.0% | noise | 50.0% | 0.0% | 50.0% | 2 |
| new-zealand-kaka | 49.8% | noise | 8.0% | 6.7% | 27.1% | 538 |
| new-zealand-kingfisher | 26.3% | noise | 10.5% | 5.3% | 34.2% | 38 |
| new-zealand-pipit | 22.2% | noise | 44.4% | 22.2% | 11.1% | 9 |
| norfolk gerygone | 39.0% | blackbird | 3.5% | 0.7% | 44.0% | 141 |
| norfolk robin | 52.2% | white tern | 4.3% | 0.0% | 36.2% | 69 |
| north-island-robin | 16.3% | noise | 6.7% | 3.8% | 54.8% | 104 |
| oystercatcher | 25.0% | None | 0.0% | 0.0% | 75.0% | 4 |
| paradise-shelduck | 40.0% | noise | 20.0% | 0.0% | 40.0% | 5 |
| parakeet | 27.0% | noise | 11.5% | 5.7% | 47.5% | 122 |
| pukeko | 64.4% | noise | 8.9% | 2.2% | 13.3% | 45 |
| red-tailed tropicbird | 12.5% | black noddy | 25.0% | 12.5% | 50.0% | 8 |
| redpoll | 25.0% | chaffinch | 5.0% | 12.5% | 42.5% | 40 |
| rifleman | 31.5% | noise | 24.2% | 5.0% | 31.0% | 400 |
| rooster | 65.7% | noise | 2.9% | 0.0% | 23.5% | 102 |
| shining-cuckoo | 60.0% | noise | 7.5% | 0.0% | 22.5% | 40 |
| silvereye | 43.4% | noise | 7.3% | 4.6% | 38.0% | 1564 |
| skylark | 45.8% | noise | 12.5% | 4.2% | 25.0% | 24 |
| slender-billed white-eye | 16.5% | silvereye | 17.7% | 1.3% | 53.2% | 79 |
| song thrush | 22.1% | noise | 11.8% | 5.9% | 46.3% | 136 |
| sooty tern | 42.9% | black noddy | 17.1% | 0.0% | 25.7% | 35 |
| south-island-robin | 19.5% | noise | 11.9% | 6.8% | 51.7% | 118 |
| southern-black-backed-gull | 33.3% | noise | 33.3% | 0.0% | 16.7% | 6 |
| sparrow | 36.4% | silvereye | 31.8% | 0.0% | 27.3% | 22 |
| spur-winged-plover | 22.2% | noise | 44.4% | 0.0% | 11.1% | 9 |
| starling | 25.0% | song thrush | 25.0% | 12.5% | 0.0% | 8 |
| tomtit | 16.9% | noise | 13.6% | 6.6% | 48.8% | 1620 |
| tree-weta | 58.8% | noise | 25.1% | 1.4% | 0.0% | 211 |
| tui | 13.5% | noise | 6.0% | 7.9% | 57.2% | 554 |
| weka | 60.4% | noise | 16.7% | 4.2% | 11.1% | 144 |
| whistler | 62.9% | fantail | 2.1% | 0.6% | 28.2% | 869 |
| white tern | 73.1% | black noddy | 3.2% | 0.0% | 15.1% | 93 |
| whitehead | 24.4% | noise | 9.8% | 2.4% | 46.3% | 41 |
| yellow-crowned-parakeet | 23.8% | rifleman | 9.5% | 0.0% | 47.6% | 21 |
| yellowhammer | 22.2% | noise | 18.5% | 11.1% | 37.0% | 27 |
| bird | 41.0% | noise | 16.9% | 15.7% | 41.0% | 83 |
| human | 49.1% | bird | 14.0% | 22.8% | 14.0% | 57 |
| noise | 66.9% | insect | 23.2% | 4.1% | 1.7% | 708 |
Fabulous work – hope to see kōkako on the next list!