Today's AI has learned to measure technique. It has not yet learned to measure art.
There is one recognised way to name the best photographs: contests with a professional jury.
Not Instagram likes. Not even "panels of professionals". A photograph earns its standing only when truly accomplished photographers judge it.
This matters because of how today's AI photo-scorers were actually built. Every one of them learned from images labelled by ordinary people — amateurs, registered users, paid annotators — never by a professional photography jury.
| System | Who judges / annotates | Expert jury? |
|---|---|---|
| Top photography contests | Selected, recognised professionals — photographers, curators, editors, art professionals | YES |
| NIMA / AVA | Registered DPChallenge users / amateur photographers; ~200 ratings per image | NO |
| LAION-Aesthetic V1 | Human raters via Simulacra, asked simply how much they liked the image | NO |
| LAION-Aesthetic V2 | Mixed human-rating datasets, including AVA / DPChallenge | NO |
| Everypixel | Trained on large stock-photo sets; algorithmic technical-aesthetic scoring | NO |
| PickScore | Real users of Pick-a-Pic | NO |
| ImageReward | Annotators trained for image-preference labelling | NO |
| HPS v2 | Human annotators ranking generated images (~798k judgements) | NO |
| VisionReward | Human annotators giving multidimensional preference labels | NO |
Almost every automatic scorer — Google's NIMA included — learned from a crowd (the AVA set), not from masters.2 A crowd feels "nice." A world jury sees what the crowd cannot. Researchers have shown models trained this way simply don't carry over to professional photography.3
"Models trained on the crowd-sourced AVA dataset behave differently on professional photographs, and do not generalize well to them."
— paraphrased from Chambe et al., Deep learning for assessing the aesthetics of professional photographs, 2022The right way is to learn from real contests, judged at the highest level.
Since 2018 we have run FOTOAWARD — an international non-profit photography contest judged in two jury rounds after an initial audience stage, whose defining feature is a jury of world-class experts.
Among the jury members are the finest photographers alive — repeat winners of the most prestigious awards in the world (World Press Photo, Wildlife Photographer of the Year, Bird Photographer of the Year, Trierenberg Super Circuit, Golden Turtle, GDT European, WPPI, etc.), National Geographic photographers, Nikon ambassadors and book authors, from ten countries. See the 2025 jury ↗
The contest itself draws winners of Hasselblad Masters, Smithsonian, HIPA and Xposure. From it comes a rare, structured record of expert taste — eight years of graded verdicts, layer by layer.
What we are looking for are the patterns in the winning work that bring these top experts to agreement. Independent agreement among judges is a measurable, scientific thing — statisticians call it inter-rater reliability.1
FOTOAWARD 2025 — winners ↗
Our own tests point clearly in one direction: today, the best available judge of artistic worth is not these scoring services, but the leading frontier AI models. This is an illustration from our daily work, not a formal study. We took ten photographs that all reached the final of our contest — every one strong — and asked three of the world's leading AI models to score each on artistic worth, 0 to 100. Then we lined those scores up against how our jury actually voted, and against the technical scorer Everypixel.
About the jury index: a raw vote count understates the signal. In the spirit of Fleiss' kappa — the standard measure of inter-rater agreement — what matters is how statistically unlikely a given level of consensus is. Of 326 works, just one drew 8 of 11 votes; three drew six. Our index combines that rarity with how close the vote came to unanimous, on a 0–100 scale. It is deliberately conservative: 8 of 11 is near-miraculous agreement, but short of a unanimous 11, so it reads 85, not 100.
Even the best AI judge available today does not reach the level of a real world-class jury.
None of this means AI can't help photographers today.
Because we study how AI reads photographs every day, we have built real tools around it — free, and already in use on our contest. Load a photograph and the system can analyse it, write a critique, suggest titles, compose a poem or a short story from it, turn it into a 19th-century engraving, or check whether it was AI-generated.
FotoMatcher — the right contest for any photograph, found by reading the photograph itself.
We have built a unique, continuously updated database covering close to every photography contest in the world. But the real work is not matching by theme or contest name: the AI studies the photograph itself and weighs how well it truly fits each of the hundreds of open contests — even when genres and categories can't tell you — plus many other factors.
It is a hard, separate project — and it is close to completion. fotomatcher.com ↗
