If you’ve paid much attention to the marketing around big flagship smartphones like the Google Pixel 2, HTC U11 or Huawei Mate 10 Pro, you might have heard the term “DxOMark” or “DxOMark Score” being thrown around.
For example, Samsung’s own hype reel about their recent-announced Samsung Galaxy S9 brags that “with a score of 99 points, we’ve earned DxOMark’s highest-ever combined photo and video rating for a smartphone camera.”
What is DxOMark? Well, here’s a quick explanation.
Who are DxO?
DxOMark is a camera ratings website that’s was originally owned by DxO Labs, an image processing company based out of France. They’re an established player in the space that makes its money selling high-end image processing software to photography enthusiasts.
They’re bonafide experts in the field so, the argument goes, if anyone can be said to be ‘qualified’ when it comes to exhaustively testing, measuring and rating the objective quality of a camera, it’s them. In their own words, they claim that “press and industry rely on DxOMark for judging lens, digital camera, and smartphone camera quality.”
Though originally focused on dedicated cameras, recent years have seen DxOMark expand into the smartphone space and, as a result, their brand and reputation has been magnified by tech companies like Google, Huawei and Samsung. Essentially, if you’re trying to launch a smartphone with a killer camera in 2018, DxOMark offer a seal of approval that tells imaging enthusiasts that you’ve got the goods.
As of September 2017, DxOMark and DxO Labs are now independent and seperate companies. According to the former, "DxOMark Image Labs is now a privately-owned, independent company. As such, we continue to pursue the development and commercialization of image quality solutions and services that support our customers in designing the best-quality camera systems for a range of markets, including smartphones, DSC/DSLRs, drones, action cams, surveillance, and automotive."
What’s a good DxOMark score look like?
Generally speaker, higher is better.
The scoring system that DxO use rates how each aspect of a phone (or camera) performs based on a comprehensive and highly-documented process. Exposure, zoom, contrast, autofocus, stabilization, noise, texture and color are all examined and measured to a oft-overwhelming level of detail. Take a look at one of their full reviews for yourself, and you’ll probably come away impressed at the extra mile and objective detail that they go into with each of their reviews.
There is, however, a handful of catches to consider here.
How much each aspect of a camera - the overall weighting - contributes to its overall DxOMark score seems to be either completely subjective or subject to a "black box" formula that the ratings site haven’t ever detailed - which is a bit of a kink in the process.
DxOMark themselves say that "The overall score is not a weighted sum of the sub-scores. It is a proprietary and confidential mapping of sub-scores into a combined score."
What’s more, DxOMark say that they often also consult with manufacturers to help them create better cameras - which presumably means ones that will score better in DxOMark’s own tests.
In addition, it’s also unclear how high the ceiling on DxOMark scores can actually go. The hype around headlines like “Google Pixel scored 99 in DxOMark” might lead you to think that their scoring system caps out at 100. However, much like the assumption that the overall score is based on a straightforward average, this isn’t actually the case
In reality, smartphone cameras scoring over 100 in DxOMark’s process is just something that hasn’t really happened up until the Galaxy S9+ showed up.
"The Overall score is also not capped at 100. That just happens to be where some of the best devices are currently," DxOmark say.
While that 104 might convey how much better DxOMark thinks that the S9+ is compared to the Google Pixel 2, it doesn’t necessarily convey the everyday user significance of that difference.
How much does having a good DxOMark score really matter?
At the end of the day, DxOMark scores might make for a great bullet point on a press release and a good general indicator of whether or not a smartphone has a decent camera. However, in the absence of open information about the way that DxOMark tally their final scores, it’s probably worth reserving about the a similar amount of salt you keep in reserve for most product reviews.
As someone who writes a lot of reviews, the amount of effort and pain-staking detail that DxOMark put into each of their comparisons is genuinely really impressive. Personally, I’ve a lot of respect for the work that they do.
That said, as tantalizing as clean, objective, numerical metrics seem, it’s difficult to back up the argument that they’re as empirical a representation of the smartphone camera experience as they might initially appear. There's a lot of value in the work that DxOmark do but they're not necessarily the be-all-end-all that the marketing often spins them as.