Cup scores are considered by some an objective measure of coffee quality, but they have been met with increasing criticism in recent years. Scoring dictates the prices producers receive, reaffirming the widely held belief that the value of coffee is more than just a numerical rating.
Despite all efforts to cup fairly and consistently, the reliance on human-led sensory assessment means decision-making is always subjective to a degree. Two Q graders may award different scores for the same coffee, for example.
Cup scores can also vary between actors across the supply chain, most noticeably between majority-consuming and majority-producing countries, highlighting broader issues of a lack of transparency and inherent bias.
I spoke with Marianella Baez Jost of Farmers Project Specialty Coffee and Krzysztof Blinkiewicz of Red Ink Coffee to learn how we can address these discrepancies in cup scores.
You may also like our article on whether cup scores mean anything to consumers.
Assessing coffee quality is inherently subjective
Cupping and scoring coffee are essential industry practices. Without protocols and tools like the SCA cupping form, there would be no standardised measures of coffee quality.
However, the idea that cup scores are the most objective way to assess coffee quality and, therefore, determine its price is inherently flawed. Individual bias is inextricably linked to human-led sensory analysis, meaning personal opinion will always be at play.
The coffee industry is global, and taste preferences vary worldwide. A flavour note that may be considered “overly fermented” by some could be perceived as highly desirable by others, for example, leading to the lack of universal calibration and shared understanding of coffee flavour and quality.
Given that supply chain actors in the Global North are largely in control of sensory assessment and scoring, these systems are often designed in their favour, failing to understand the flavour experiences of people in the Global South. To address this directly, industry professionals in countries such as Taiwan and Indonesia have developed localised Flavor Wheels that promote more diverse and inclusive conversations about coffee quality analysis.
Despite this progress, the sensory assessment protocols designed by actors in majority-consuming countries remain the universally accepted methods to evaluate coffee quality. This has a major impact on pricing mechanisms, meaning actors in the Global North have more control over the prices producers receive for their coffee.
Addressing discrepancies across the supply chain
Personal bias can mean that two Q graders may award the same coffee two different scores. While this is much less likely to happen in a quality control lab where Q graders regularly calibrate, for instance, it’s not an uncommon occurrence across various levels of the supply chain.
In some cases, noticeable discrepancies may exist between the scores provided by producers, exporters, importers, and roasters. This can occur for several reasons, but it’s problematic in a number of ways.
For instance, a producer may cup their coffee and award it 87 points, sharing this number with the exporter or importer, which in turn dictates the differential paid above the market price. However, the importer or roaster may score it 85 points, leaving farmers vulnerable to price negotiations.
In another case, a roaster may score a coffee higher than the producer. If they choose to display this rating on their packaging, they can charge a premium that may not always be passed on to producers.
This is known as “pointwashing,” or, in cases where roasters feel pressured to outperform their competitors and score their coffees higher, we can refer to it as “cup score inflation.”
“Pointwashing is the inflation or misrepresentation of coffee scores, whether intentional or accidental, to give a coffee more value than it actually has,” says Krzysztof Blinkiewicz, the founder of coffee education platform Red Ink Coffee, Authorised SCA Trainer, and Q grader.
“It occurs when the score printed on a bag or used in marketing no longer (or never did) reflect the real sensory experience, especially from the consumer’s perspective,” he adds. “This is a systemic issue, not just a one-time exaggeration. It often stems from a lack of calibration, a lack of transparency in the supply chain, or pressure to meet market expectations.”
Understanding the reasons behind “pointwashing”
As the coffee industry relies heavily on cup scores and quality assessment to set prices, there is a clear need to understand why discrepancies exist, especially between majority-producing and majority-consuming countries.
The most important factor to note is that coffee quality changes as it’s transported across the supply chain. From harvesting to drying, milling, and export, each stage of the value chain affects flavour and quality. The coffee that a quality control lab manager cups in Kenya won’t taste exactly the same as what the importer in Germany receives, for instance.
Many of these changes are unavoidable. With geographical distance and time frames in mind, it’s inevitable that cup scores will differ across countries. However, the main problem lies in the fact that buyers are largely in control of scoring systems, which means they are more easily able to influence prices.
“Pointwashing often happens passively. People in the supply chain, primarily roasters and baristas, take the score and use it without thinking about where it came from or how it was calibrated,” Krzysztof says. “Still, there are clear cases where importers or roasters inflate scores to justify a higher price or to stand out in a competitive market.
“When ratings become a tool of persuasion rather than for assessing quality, the integrity of the entire value chain begins to deteriorate. It also affects the overall quality of coffee on the market,” he adds. “Producers may feel compelled to ‘improve’ coffees with descriptions that don’t match the actual taste, as well as with processing methods, including experimental techniques, to ‘justify’ defective flavours with inflated scores.
“Roasters may lose the ability (or even the desire) to independently evaluate flavour, relying instead on inherited ratings, or their internal quality control becomes permanently affected by the error of suggestion.”
Even though they carry little meaning for most consumers, inflated cup scores have an impact at the end of the supply chain if they are used for marketing purposes.
“Consumers are misled. They may drink something unpleasant but praise it anyway because of the label rating and the (often significantly) higher price that comes with it,” says Krzysztof.
What are the solutions?
Cup scores often divide opinion in specialty coffee. While some assert they are the most objective way to define quality, others believe they are too narrow and don’t factor in a coffee’s holistic value.
New quality assessment protocols claim to address this issue, but many in the industry question their motives and the effectiveness of their implementation. Moreover, they’re unlikely to address subjectivity and discrepancies in cup scores, especially between majority-consuming and majority-producing countries.
“Regardless of whether pointwashing is done with good or bad intentions, the result is the same: distortion of value, breakdown of trust, and confusion about the true meaning of quality,” Krzysztof says.
For some, the answer is to withhold personal cup scores from buyers, avoiding any potential influence or persuasion in their decision-making.
“My opinion on cup score discrepancies is simple, and I have applied it to my business from the start: For Farmers Project Specialty Coffee, the focus is on the quality of the whole process, from seedling to soil inputs to picking to wet and dry milling,” says Marianella Baez Jost, a producer at Café con Amor in Costa Rica. “We conduct our internal quality control using Q graders in Costa Rica and the US, but we never disclose scores to customers. We give them the samples and let them be the final judge.”
As consumer demands shift and they become increasingly price-conscious, many favour highly customised drinks that offer perceived value for money. Roasters, meanwhile, are grappling with tight margins, making cost-effective coffees more viable. Both reshape buying behaviour and affect the type of coffees that roasters are sourcing.
“Our industry is currently selling more sugar than coffee. Cold coffee is growing fast, and drink menus are evolving faster,” Marianella adds. “Roasters source coffee thinking of what their average customer likes – what coffees can be a solid addition, a brilliant, fun, or unique release?
“They are ultimately not buying based on cup score alone, but on flavour quality, and banking on consistency, taste character, and a clean cup.”
The need for more transparency
Conversely, a push for even more transparency, often considered a cornerstone of specialty coffee, could help resolve notable differences in cup scores.
“Every participant in the supply chain should be encouraged (or ideally required, although this isn’t realistic in the current global context) to document how the score was determined,” Krzysztof says. “Who tasted the coffee, when, where, and under what conditions? Which cupping form was used? Was calibration performed? What were the results of any comparisons between the place of origin and the consumer market?”
The issue, however, lies in ensuring that coffee professionals at origin have equal access to similar quality control and analysis resources as those in majority-consuming markets. Many smallholders are unable to formally cup and taste their own coffees, which hinders their ability to influence the prices they receive.
“If we want to communicate evaluation results, we need to be completely honest about their origin and context,” Krzysztof says. “Ideally, each score would be accompanied by metadata, maybe even linked through a code to a reliable source. Some even suggest using blockchain.”
AI-driven technology offers a solution to reduce the quality differentiation between buyers and sellers, but ensuring equal access across geographies would still present a barrier.
Ultimately, the solution could lie in better transparency about how cup scores change over time.
“If we offered one score per level of the supply chain, for example, one from the producer’s QC lab, one from the importer, and one from the roaster, it could help consumers understand how the coffee evolves,” Krzysztof says. “But how many consumers would actually know what that all means? People want clarity, not complexity.”
Human-led sensory assessment is currently the universally accepted method for defining coffee quality, and therefore, its price. Inevitably, this means bias will always be an influence, leading to differences in cup scores between individuals across the supply chain.
While technology offers solutions to this issue, they aren’t yet widely available. For now, the importance lies in understanding how coffee quality changes over time, as well as how producers and professionals at origin need to be more involved in the quality control and pricing process, rather than simply being “price takers”.
Enjoyed this? Then read our article on what cupping can tell producers about their coffee.
Perfect Daily Grind
Want to read more articles like this? Sign up for our newsletter!
