Information Architecture and User Flows Questions

Direct answer

Run an unmoderated remote tree test, a study where participants see only the navigation labels stripped of any visual design, and click their way to where they think a task lives, with no researcher present, so results reflect the labels and hierarchy alone. Track task success rate as the primary signal for whether the structure itself is right, and time-to-find as a secondary signal for how much friction is in getting there; treat a full IA redesign as justified only when a critical task both fails often and fails the same way for many people, and treat a quick relabel as the fix when people eventually succeed but take a long detour to get there.

Structured elaboration

Task design

• Write 6-8 realistic, single-goal tasks in the user's own language, never in navigation-menu language ("Change who can see your profile," not "Find the Privacy tab"), since the point is testing whether the labels match how people actually describe the goal.
• Randomize task order per participant to avoid a learning effect where later tasks get easier just from having explored the tree already.
• Recruit enough participants that a pattern, not one person's fluke click, is what you are reading; a minimum of around 30-50 completes per task is a reasonable floor for a first read, more for tasks you plan to make a big structural call on.

Success metrics

• Task success: did the participant land on the correct node.
• Time-to-find: how long from task start to the first click on the correct node.
• Directness: did they get there on the first click, or wander through wrong branches first.
• First-click data: which wrong node people hit most often, since a shared wrong first click across many participants is the strongest structural signal in the whole test.

Worked example

Suppose the unmoderated test runs with 60 completes across three tasks and reports:

Task 1 is fast and high-success: the label and its location both work, ship as-is. Task 3 is fast and reasonably high-success: fine, maybe worth a small relabel if time allows but not urgent. Task 2 is both slow and low-success, the pattern that matters most here: people are not just wandering, a large share of them are landing in the wrong place entirely, which points at the label living under the wrong parent category rather than just being worded oddly.

Reading time-to-find against success rate when they disagree. High success paired with a long time-to-find usually means the destination is right but the label does not match the words people search for first; they get there eventually through trial and error, so this calls for a relabel, not a rebuild. Low success paired with any time reading, fast or slow, means people are confidently choosing the wrong branch, which is a structural problem: the concept lives in the wrong place in the tree, and no amount of rewording the label where it currently sits will fix that.

When the pattern justifies a full redesign versus a quick relabel. Redesign the structure when a critical or frequent task has a low success rate (in the worked example, Task 2's 58% is well below what you'd want for a task most users need to complete) and a large share of the misses cluster on the same wrong node, indicating the concept sits in the wrong branch altogether. Ship a quick relabel instead when success is already high but time-to-find or directness is poor, since that means the destination is correct and only the words pointing to it need to change.

Trade-offs and pitfalls

An unmoderated test cannot tell you why someone chose a wrong node, only that they did; pair a low-success finding with a short follow-up moderated session or an open-text "why did you click there" prompt before committing to a full redesign based on click data alone. Testing labels in isolation from visual design also means a genuinely bad visual hierarchy on the live site could still sink a label that tested well in the tree test, so a tree test result is necessary evidence, not sufficient proof, before shipping a structural change.

Direct answer

Run an unmoderated tree test against a stripped, text-only version of the proposed hierarchy, recruit across the personas who actually rely on the help center, write tasks as real goals rather than instructions that name the answer, and use the gap between success rate and time-to-find, plus which paths people actually take, to decide whether the problem is a wrong label or a wrong structure.

Structured elaboration

Recruitment and personas

• New users (their first 0 to 3 months), who lean on help articles most.
• Power users or admins, who look for advanced or less-common topics.
• Internal support staff, who bring domain knowledge about where confusion actually happens.

Sample size
30 to 50 total participants (roughly 10 to 15 per persona) balances a usable quantitative signal against recruiting cost, appropriate for validating or reworking a taxonomy rather than shipping a single high-stakes decision off it alone.

Example tasks (help center)
Write tasks as goals, never as instructions containing the answer:

• "How do you reset your password if your account is linked through single sign-on?"
• "Where would you look to find API rate limits?"
• "How would you transfer billing ownership to someone else in your organization?"

Success and failure criteria
Success: the participant lands on the correct node within a small number of clicks (commonly 3) and rates it as clearly relevant. Failure: a wrong node, giving up, or exceeding a time limit (around 90 seconds).

Metrics
Success rate per task and per node, time to find (first click and total time), click depth and backtracks, path diversity (how many distinct routes people took to reach the same correct node), and a post-task confidence rating.

Analysis
Quantitatively, flag any node under roughly 60% success and visualize dominant versus divergent paths (a path diagram makes clusters of wrong turns visible at a glance). Qualitatively, cluster the comments people leave to separate "the label confused me" from "the category was too deep" from "I expected it somewhere else entirely."

Next steps
Quick wins (relabeling an ambiguous node, adding a cross-link) can ship immediately; structural changes (merging or splitting categories) should be re-validated with a second, smaller tree test before committing.

Worked example two: a news website's sitemap

The same method transfers to a very different content domain, where task phrasing changes shape because the content is organized by recency and topicality rather than by troubleshooting a specific action. Example tasks here look more like "Find today's top story about the upcoming election" or "Find the correction notice for an article that ran last week" or "Find the opinion section for this newspaper," rather than the how-do-I phrasing a help center uses. Success and failure criteria stay the same (reach the right node, within a click budget, rated relevant), but the failure pattern tends to differ: people more often fail by choosing a topical category (Politics, Business) when the right node was actually organized chronologically (This Week, Archive), a structural mismatch a help center rarely produces since help content is almost always organized by task rather than by date.

Trade-offs and pitfalls

• A task phrased too close to the label ("Find the Billing page") tests reading comprehension, not findability; keep tasks in the user's own goal language.
• Success rate and time-to-find can disagree (a fast wrong answer, or a slow correct one); when they do, treat it as a sign the label got someone to a plausible-looking wrong node quickly, which is often worse than an honest, slower failure.

Plan summary (30 min moderated, low-fi wireframes)

• 5 min: intro, consent, demographics
• 20 min: tasks (think-aloud), probing
• 5 min: debrief & SUS-like quick rating (SUS is the System Usability Scale, a standard 10-question usability survey)

Participant criteria

• 6–8 participants (first-time users)
• Age 18–55, mix of novice/occasional tech users
• No prior account on our product; interested in category
• Diverse accessibility needs if possible

Tasks (with success criteria)

1. “Create an account”: start from homepage, sign up with email.
   Success: completes sign-up flow and lands in profile area.
2. “Verify email and sign in”: locate verification step and sign in.
   Success: able to confirm where/when to verify.
3. “Complete profile”: add name, photo, preferences, and save.
   Success: fills required fields and sees confirmation.
4. “Find help”: if stuck, locate help or undo option.
   Success: finds support/FAQ or cancel/edit affordance.

Success metrics

• Task completion rate (complete, partial, fail)
• Time on task per task
• Errors encountered and severity
• Time to first action (click)
• Post-task confidence (1–5) and overall SUS-ish rating

Key script prompts

• “Please think aloud as you go.”
• Neutral nudge: “What are you looking for right now?”
• If stuck: “Can you tell me what you expect to happen next?”
• Probing after task: “What caused hesitation?” “What would make this easier?”

Capture & prioritization

• Record screen + audio, moderator notes, timestamped usability issues, and task success flags.
• Tag issues by frequency, impact (prevents completion / slows task / cosmetic), and effort to fix.
• Prioritize using RICE-like lens: Reach (how many users), Impact (on task success), Confidence (data), Effort (dev estimate).
• Deliverables: annotated wireframes with issues, top 5 prioritized fixes, and quick wins for next sprint.

Direct answer

Use open card sorting early, while you still need to discover how people actually think about the content, and closed card sorting once a draft structure exists and needs checking; recruit a mix that matches who will actually use the final structure, and convert the raw sort data into a taxonomy through clustering, consensus scoring, and a deliberate label pass, not a straight read of whichever grouping got the most votes.

Structured elaboration

When to run which

• Open card sort: early discovery, when a content set has never been organized or an existing structure is clearly failing (rising support-ticket volume about not finding things, for example) and you need to see the categories and names people invent on their own.
• Closed card sort: once you have a candidate taxonomy (from an open sort, stakeholder input, or an existing structure you suspect is wrong), to test whether people place items where you expect and flag which ones do not.

Participants and diversity

• Open sort: 20 to 30 participants, since new categories tend to stabilize in that range.
• Closed sort: 30 to 50, for a more confident read on per-item placement.
• Recruit across the people who will actually use the structure: new users, frequent users, and, importantly, the support agents who field the questions the structure is supposed to prevent, not just one segment.

Analysis
Cluster items by how often participants grouped them together (a similarity or co-occurrence matrix, then hierarchical clustering), and compute a consensus score, the percentage of participants who agreed on an item's placement. Flag any item under roughly 60% consensus for a closer look rather than forcing it into whichever cluster narrowly won.

From sort output to a working taxonomy

1. Map the stable clusters to candidate top-level categories and subcategories.
2. Normalize labels: merge synonyms, and prefer the language participants actually used over internal jargon.
3. Write explicit placement rules for ambiguous or multi-fit items (a "billing" article that is also a "troubleshooting" article needs a documented tie-breaker, not a coin flip).
4. Validate the draft with a closed card sort or a tree test before committing it to navigation.

Worked example: the same method on a BI (business intelligence) report catalog

The same steps apply outside a help center. A BI product with 80 named reports (Customer Churn Report, Pipeline Velocity, Cohort Retention, and so on) needs a category structure too, and the participant mix matters even more here: BI analysts already carry internal naming conventions that may not match how the executives who consume the reports actually think about them. Recruiting analysts, a handful of executive stakeholders, and end-of-report consumers together for the same open sort surfaces the mismatch directly: analysts might group by data source ("Salesforce reports," "product-analytics reports") while executives group the same items by business question ("Are we growing," "Are we retaining customers"). Whichever grouping the closed sort later validates as findable is the one that ships, even if it is not the one the team that built the reports would have chosen on its own.

Trade-offs and pitfalls

• Different segments (new users versus support agents, analysts versus executives) can produce genuinely conflicting groupings; do not average them away silently, report the split and make a deliberate call about which audience the primary navigation should serve.
• Stakeholders often want a quantitative-looking readout; consensus scores and cluster diagrams give you that, but pair them with two or three verbatim participant quotes so the "why" survives the summary.

Approach summary (role: UX Designer)
I’d combine complementary methods: moderated task-based testing, tree testing, first-click testing, plus unmoderated remote sessions for scale and analytics review.

Methods & rationale

• Task-based moderated testing (desktop + mobile): observe real behaviour, probe when users hesitate, test responsiveness and mobile patterns (hamburger, bottom nav).
• Tree testing: validate information architecture independently of UI to see where users expect items to live.
• First-click testing: measure where users try to start when solving a task (critical for navigation success).
• Unmoderated remote + analytics: gather larger N for time-on-task, paths, drop-offs.

Metrics to collect

• Task success rate (binary + partial success)
• Time on task
• First-click correctness (%)
• Path efficiency (steps taken vs optimal)
• Error types and frequency
• Task abandonment rate
• SUS (System Usability Scale, a standard 10-question usability survey) or a single-item ease rating + qualitative confidence/comments

Ensuring realistic discovery tasks

• Source tasks from analytics (top search terms, drop-off pages) and support logs to mirror real needs.
• Build personas and scenarios (e.g., “You need to find warranty info before buying a laptop for a gift”).
• Phrase tasks as goals, not UI instructions; avoid hinting where to click.
• Include mobile-specific contexts (on-the-go, one hand) and environmental constraints.
• Pilot tasks with colleagues to check clarity and realism, iterate before formal sessions.

Example task phrasing: “You’re buying a laptop and want to check warranty terms: find where to view warranty details and tell me what you’d expect to find there.”