iNaturalist is a website that crowdsources pictures of plants and animals to help identify species. Its tagline is “A Community for Naturalists.” iNaturalist is administered by its own small charit…
So are image classifier models. They were terrible for several years, and eventually improved. LLMs are pretty good at retrieval augmented generation, which is probably the whole idea.
A lay person takes a picture of a beetle. They want to know what it is.
An image classifier correctly identifies it as a long horn, wood boring beetle. It has 5 species with a greater than 80% probably.
Human written and curated taxonomical descriptions are pulled out of a database.
An LLM interprets the complicated language of taxonomy, defining terms and asking the user plain language questions about the beetle.
Maybe this makes the whole process more accessible for lay users. Maybe it helps people understand what questions to be asking for identification. I mean, I’m just guessing at the implementation, but it seems pretty logical to me.
So are image classifier models. They were terrible for several years, and eventually improved. LLMs are pretty good at retrieval augmented generation, which is probably the whole idea.
A lay person takes a picture of a beetle. They want to know what it is.
An image classifier correctly identifies it as a long horn, wood boring beetle. It has 5 species with a greater than 80% probably.
Human written and curated taxonomical descriptions are pulled out of a database.
An LLM interprets the complicated language of taxonomy, defining terms and asking the user plain language questions about the beetle.
Maybe this makes the whole process more accessible for lay users. Maybe it helps people understand what questions to be asking for identification. I mean, I’m just guessing at the implementation, but it seems pretty logical to me.