iNaturalist is a website that crowdsources pictures of plants and animals to help identify species. Its tagline is “A Community for Naturalists.” iNaturalist is administered by its own small charit…
It is my understanding that the advances in classifier models were and are inexorably linked to generative models. Wasn’t Deepdream a fairly crude inversion of existing classifier models?
You’re totally misunderstanding the context of that statement. The problem of classifying an image as a certain animal is related to the problem of generating a synthetic picture of a certain animal. But classifying an image of as a certain animal is totally unrelated to generating a natural-language description of “information about how to distinguish different species”. In any case, we know empirically that these LLM-generated descriptions are highly unreliable.
So are image classifier models. They were terrible for several years, and eventually improved. LLMs are pretty good at retrieval augmented generation, which is probably the whole idea.
A lay person takes a picture of a beetle. They want to know what it is.
An image classifier correctly identifies it as a long horn, wood boring beetle. It has 5 species with a greater than 80% probably.
Human written and curated taxonomical descriptions are pulled out of a database.
An LLM interprets the complicated language of taxonomy, defining terms and asking the user plain language questions about the beetle.
Maybe this makes the whole process more accessible for lay users. Maybe it helps people understand what questions to be asking for identification. I mean, I’m just guessing at the implementation, but it seems pretty logical to me.
It is my understanding that the advances in classifier models were and are inexorably linked to generative models. Wasn’t Deepdream a fairly crude inversion of existing classifier models?
You’re totally misunderstanding the context of that statement. The problem of classifying an image as a certain animal is related to the problem of generating a synthetic picture of a certain animal. But classifying an image of as a certain animal is totally unrelated to generating a natural-language description of “information about how to distinguish different species”. In any case, we know empirically that these LLM-generated descriptions are highly unreliable.
So are image classifier models. They were terrible for several years, and eventually improved. LLMs are pretty good at retrieval augmented generation, which is probably the whole idea.
A lay person takes a picture of a beetle. They want to know what it is.
An image classifier correctly identifies it as a long horn, wood boring beetle. It has 5 species with a greater than 80% probably.
Human written and curated taxonomical descriptions are pulled out of a database.
An LLM interprets the complicated language of taxonomy, defining terms and asking the user plain language questions about the beetle.
Maybe this makes the whole process more accessible for lay users. Maybe it helps people understand what questions to be asking for identification. I mean, I’m just guessing at the implementation, but it seems pretty logical to me.