Card sorting can be either qualitative or quantitative, and it’s often both. The method itself is flexible enough to produce rich, exploratory insights about how people think or hard statistical patterns about how they group information. What determines the nature of your data is how you design the study: the type of sort you choose, the number of participants you recruit, and how you analyze the results.
Open Sorting Leans Qualitative
In an open card sort, participants group items into whatever categories make sense to them and name those categories themselves. There are no predefined buckets. This makes it a naturally qualitative method because the output is exploratory: you’re discovering how people mentally organize information rather than testing a hypothesis. Participants have full flexibility to express how items are grouped in their minds, and the category labels they create give you additional data about their vocabulary and reasoning.
Open sorts are especially useful early in a project when you’re trying to uncover natural groupings and mental models. Researchers often ask participants to think aloud during the sort, explaining why they placed certain cards together. That verbal data is purely qualitative. It tells you not just what people grouped together but why, which is critical for building an information architecture that feels intuitive.
Closed Sorting Leans Quantitative
A closed card sort gives participants predefined categories and asks them to place each card into the one that fits best. Because everyone works with the same set of categories, the results are easy to compare across participants and lend themselves to statistical analysis. You can calculate how often each card lands in a given category, measure agreement rates across participants, and determine whether your proposed structure holds up.
Closed sorts work well when you already have a category structure and want to validate it. The data is inherently more standardized, which makes it easier to run with larger sample sizes and draw conclusions about patterns in the broader population.
Hybrid Sorting Combines Both
A hybrid card sort starts with predefined categories but lets participants create new ones if nothing fits. This blends the quantitative structure of a closed sort with the exploratory flexibility of an open sort. You get measurable placement data for your existing categories while also learning where your structure has gaps, because participants will invent categories for cards that don’t belong anywhere obvious.
How Sample Size Shapes the Data
The number of participants you recruit determines whether your card sort produces qualitative insight or quantitative evidence. Nielsen Norman Group recommends at least 15 participants for a qualitative card sort, where the goal is understanding the reasoning behind groupings. For a quantitative study where you need results that generalize to a broader population, the recommendation jumps to 30 to 50 participants.
A qualitative card sort with 10 people might reveal three or four distinct mental models your users hold. A quantitative card sort with 40 people can tell you that 78% of participants placed “shipping policy” under “Orders” rather than “Help,” giving you statistical confidence in that placement. Both are valuable, but they answer different questions.
Quantitative Outputs From Card Sorting
Even open card sorts produce quantitative data once you have enough participants. Modern card sorting tools generate several types of statistical output automatically.
- Similarity matrices show what percentage of participants grouped any two cards together. The higher the percentage, the stronger the association between those items. Cards that 80% of participants pair together clearly belong near each other in your navigation.
- Dendrograms are tree diagrams that visualize how cards cluster based on sorting behavior. They show hierarchical relationships, letting you see which items form tight groups and which sit loosely between categories.
- Agreement rates measure how consistently participants placed a card into the same category. High agreement means the placement is intuitive. Low agreement signals that a card’s home in your structure isn’t obvious.
Tools like Maze, OptimalSort, UXtweak, and UXMetrics generate these analyses automatically, producing similarity matrices, auto-grouped categories, frequency rankings, and downloadable reports with visualizations. You don’t need to crunch the numbers by hand.
Qualitative Outputs From Card Sorting
The qualitative side of card sorting shows up in three places. First, the category names participants create during open sorts reveal the language real users use, which often differs from internal jargon. Second, think-aloud comments during moderated sessions explain the logic behind groupings. Third, the patterns of disagreement between participants can surface fundamentally different mental models, where one group of users thinks about your content by task and another thinks about it by topic.
These insights don’t reduce to a single number. They require interpretation, pattern recognition, and judgment. That’s what makes them qualitative, and it’s also what makes them so useful for designing navigation that matches how people actually think.
Choosing the Right Approach for Your Project
If you’re building a new website or app and don’t yet have a category structure, start with a qualitative open sort. Use 15 to 20 participants, run it moderated so you can ask follow-up questions, and focus on discovering the natural groupings and labels your users expect. This gives you a draft structure grounded in real mental models rather than internal assumptions.
If you already have a structure and want to test whether it works, run a quantitative closed sort with 30 to 50 participants. Look at agreement rates and similarity matrices to identify where users align with your categories and where they struggle. Cards with low agreement need to be renamed, moved, or split across categories.
Many teams run both in sequence: an open sort to generate a structure, then a closed sort to validate it. This treats card sorting as a mixed-methods technique, pulling qualitative insight and quantitative confirmation from the same basic activity. The method is neither purely one nor the other. It’s both, depending on what you need it to be.