Knowledge Sharing and Team Enablement Questions

Direct answer

Teach judgement, not tools. Non-technical staff do not need to know how a model is trained; they need to know what it is good at, where it fails, what must never be pasted into it, and how to decide whether an AI use is safe for their job. I would build one short shared core (90 minutes), role-specific labs for product, legal and sales built from their real decisions, and a one-page decision checklist. I would pilot it as a staggered rollout (the groups start training at different dates) with a waiting group as comparison, and set the success bar before the pilot starts.

Content: what each audience needs

Shared core concepts: a language model produces likely text from patterns, not verified facts; the same prompt can give different answers; quality must be measured on examples, not by demo; bias and privacy are risks; and "when not to use AI" is as important as "how".

Programme shape

• 90-minute core workshop with live examples of failures.
• 60-minute role lab where each group works its own scenarios.
• A one-page checklist: what data goes in, what happens if the answer is wrong, who reviews it, is it approved, how would we notice a failure.
• A monthly 15-minute open office hour with the AI team.

Pilot

Pick 30 people who reflect each function, not just volunteers (volunteers are already enthusiastic and skew results). Assign them to two cohorts (groups that go through the programme together) of 15 at random within each function, for example by drawing names, so that neither managers nor volunteers choose who trains first (a hand-picked cohort A would make any gain reflect who was picked, not the programme): cohort A trains now, cohort B trains four weeks later and serves as the comparison meanwhile. Both take the same 10 scenario questions (rephrased between rounds so memorisation does not help) before and after. Also track behaviours: unrealistic requests sent to the AI team, and legal review flags on AI-related material.

Worked example (illustrative numbers)

Scenario item: "A model summarises a contract and cites clause 14. What do you do before relying on it?" Correct: verify clause 14 in the source, because the citation may be invented.

• Cohort A: 4.0 out of 10 before, 6.5 after, a change of 6.5 - 4.0 = 2.5.
• Cohort B (not yet trained): 4.1 before, 4.4 at the same time, a change of 0.3.
• Effect over background change: 2.5 - 0.3 = 2.2 points.

Reading 2.2 points. On a 10-question test it means the trained group got about two more questions right than background change would explain. With only 15 people per cohort, averages wobble: if individual scores typically vary by about 2 points, an average of 15 people wobbles by roughly 2 / sqrt(15) = 0.5, and the gap between two such averages by roughly 0.7 points. So 2.2 is about three times that noise, which is convincing; a gap of 0.5 would not be.

Decide up front (a convention, not a law), for example: "scale if the trained group gains at least 2 points more than the waiting group and the behaviour measures do not worsen; if the gap is between 1 and 2, extend the pilot to more people; below 1, redesign".

Trade-offs and pitfalls

• Leaders often ask for "prompt tricks". Resist; tricks age in months, judgement lasts.
• Fear-heavy training makes people avoid useful tools, hype-heavy training makes them careless. Show both wins and failures.
• Content goes stale as tools change: assign an owner and review it every quarter.
• Scores measure knowledge, not behaviour, so keep the behavioural signals in the pilot.

Direct answer

Make writing the hypothesis the entry ticket to an analysis: no query runs until the request states the decision it serves, a hypothesis, and what result would change the decision. Back it with a short review before the work begins, a scorecard after it ends, and leadership that celebrates refuted hypotheses. To show it took hold, track the share of analyses with a hypothesis recorded before the first query, and look for a healthy refutation rate and decisions that actually changed.

What I would put in place

1. A request template with five fields: the decision, the hypothesis ("we think the drop is concentrated in mobile checkout"), the evidence that would support it, the evidence that would refute it, and what we would do in each case.
2. A 10-minute hypothesis review before work starts, with a peer asking "what would prove this wrong?"
3. A post-analysis scorecard: confirmed, refuted or inconclusive, plus what we learned. Refuted results are logged as valuable.
4. Coaching in the work itself: leads ask "what is your hypothesis?" in stand-ups and reviews, not in a training deck.
5. Guarding against HARKing (hypothesising after results are known, meaning writing the hypothesis to match what you found): the hypothesis field is timestamped and compared to the first query in the query log.

Worked example (illustrative numbers)

A ticket: "Why did trial signups drop?" becomes:

Decision: whether to pause the paid campaign. Hypothesis: the drop is concentrated in mobile users arriving from the campaign. Supports: mobile share of the drop above 70%. Refutes: mobile drop close to desktop drop. If supported: pause mobile spend. If refuted: look at pricing page changes.

Adoption measure, defined precisely:

Month 1: 6 of 24 closed analyses, which is 25%. Month 3: 20 of 25, which is 80%. Companion signals: refutation rate and the count of analyses that changed a decision.

There is no magic target for refutation rate, but a healthy record is neither near 0% nor near 100%. Example scorecard for month 3 (25 closed analyses, 20 with a hypothesis logged first): among those 20, 8 confirmed, 9 refuted, 3 inconclusive, so 9/20 = 45% refuted. If the same team showed 0 refuted out of 20 for several months, I would audit the hypotheses for vagueness or after-the-fact writing.

Example scorecard row: "Ticket 412 | Hypothesis: drop concentrated in mobile campaign traffic | Result: refuted (mobile drop 12%, desktop 11%) | Learned: not a channel problem, check pricing page | Decision changed: no mobile pause".

Trade-offs and pitfalls

• Mandating the template can produce box-ticking ("hypothesis: X will change"). Audit five tickets a month for whether a result could really have refuted the hypothesis.
• Exploratory work has no hypothesis yet. Allow a labelled "exploration" track with a time limit, or analysts will fake hypotheses. Example: "Exploration, 2 days: profile the new events table to see what is there", which ends in a written hypothesis for the next step or a decision to stop.
• Speed complaints are real: keep the template to five short lines.
• The rate proves habit, not quality. Pair it with outcomes such as decisions changed and a peer-rated sample of analyses.

Direct answer

Treat influencing and cross-functional communication as a small set of trainable behaviours, not a personality. First diagnose which behaviours are missing by interviewing the stakeholders on the receiving end. Then teach with practice on real material (role-plays, shadowing, stakeholder interviews, reviewed written artifacts) over months, and evaluate by changes in stakeholder feedback and decision outcomes, not by attendance.

Structured elaboration

1. Diagnose (weeks 1 to 3). Interview 6 to 10 stakeholders (product, sales, operations, leadership) with the same questions: When has this team's advice helped you decide? When has it confused you? Turn the answers into three or four target behaviours, for example:

• Lead with the decision and recommendation, then the evidence.
• Translate technical results into the stakeholder's terms (cost, risk, time, customer effect).
• Handle pushback without retreating or over-arguing.
• Ask what the stakeholder needs before presenting.

2. Delivery methods

• Stakeholder interviews done by participants themselves, to learn how others decide.
• Role-plays using anonymised real past disagreements, with a coach playing the sceptical stakeholder. Record and review one round.
• Shadowing a product manager or senior engineer in a stakeholder meeting, then debriefing what they chose to say and skip.
• Written artifact review: participants write a one-page decision memo (recommendation, options, risks) and get feedback against a rubric.
• Run it as monthly sessions over about six months with practice in between, because behaviour changes through repetition.

3. Evaluation. A common structure is four layers: reaction (did they find it useful), learning (can they do it in an exercise), behaviour (do they do it at work), and results (do decisions get better). Measure the last two:

• The same three-question stakeholder rating before and after (1 to 5 scale each; the worked example below tracks one of the three questions, "the recommendation was clear and usable").
• Memos scored blind against the rubric, before and after.
• Decision cycle time: days from a recommendation to a decision.

Worked example

Eight participants, each rated by their stakeholders on "the recommendation was clear and usable" (1 to 5). Before: 2, 3, 3, 2, 3, 4, 2, 3. The sum is 22, mean 22/8 = 2.75. After six months: 3, 3, 4, 3, 4, 4, 3, 4. The sum is 28, mean 28/8 = 3.5. That is a rise of 0.75 points. With only eight people and no comparison group, treat it as encouraging, not proof: a waitlist cohort trained later would show whether the rise came from the programme.

Trade-offs & pitfalls

• Generic soft-skills workshops without real cases produce no transfer. Use the team's own stakeholders and disagreements.
• Manager buy-in: if managers do not create chances to present to stakeholders, the training has nowhere to land.
• Gaming: stakeholders rating friends generously. Use anonymous collection and compare against a later cohort.
• Stronger is not louder. Teach listening and framing, not persuasion tricks.

Direct answer

I use a shared core plus role-specific tracks. Everyone gets the same short foundation in how the team works (so practices converge), and each person gets a path that fills the gap between their background and our standard. A diagnostic on day 1 decides what goes in each track, and shared first projects mix backgrounds so the practices spread through working together.

Structure

1. Shared core (about the first week, same for everyone): how we ship (version control is the shared history of code changes, for example git; code review means a teammate reads and approves your change before it merges; definition of done is the team's written checklist for when work counts as finished, such as tested, reviewed and documented), how we handle data and reproducibility (anyone can rerun your analysis and get the same result), team values of collaboration and ownership, who owns what, and the product context.
2. Day-1 gap map: a short self-assessment plus a small exercise, marking skills as strong / can do with help / new. This decides the track, not the job title. Sample items: "I can open a pull request and respond to review comments" (strong / help / new); "I can write a SQL query joining three tables"; "I can explain why a model's accuracy could be misleading on rare events"; "I can package code so a teammate can install and run it". The small exercise is, for example, fixing a failing test in a sandbox repository, which shows in 30 minutes what a title cannot.
3. Role tracks (weeks 2 to 6): aim at the typical gap. Example week 1 for everyone: Monday, tour of the product and who owns what; Tuesday, set up the laptop and merge a trivial change through code review; Wednesday, walk one existing analysis end to end and rerun it; Thursday, read two recent definition-of-done examples; Friday, gap map review with the manager. Track content then differs (see the table).
4. Shared project: a small real deliverable in mixed pairs (for example a researcher with a software engineer), reviewed by the whole team.
5. Checkpoints at 30 and 90 days against the same expectations for all.

Worked example (illustrative): three hires

For teams spanning several product teams (such as a business intelligence group), add a technical curriculum (for example how the warehouse tables are modelled, which dashboard and query tools we use, how to request data access) and a business curriculum (for example each product team's goals, the five metrics it reports and what its jargon means, taught by that team's lead in a 45-minute session), and a hands-on project per product area so knowledge is applied not just heard.

The first shared project: build and ship a small metric pipeline (an automated job that pulls data, computes a business number such as weekly active users, and publishes it). The researcher and analyst each pair with the software engineer for code, and the software engineer pairs with the researcher on evaluation. Everyone ends up exposed to the shared practices.

Trade-offs and pitfalls

• Too many separate tracks fragments the team. Keep the core non-negotiable and the tracks short.
• Do not label anyone "the researcher" and skip the code review training; unequal standards are how quality splits.
• Cost: designing tracks takes upfront effort. Start with the two largest background groups, and add tracks as hires arrive.

Direct answer

I would hand over a short, prioritised package that lets the next person re-run my work and, just as important, tells them what I do not know. Five things, in this order: (1) how to reproduce, (2) where the results are, (3) what I believe and why, (4) what is still open, (5) who to ask. Each one exists because its absence costs the new person days.

The five elements, each with its reason

1. Reproduction recipe. The exact code version (a git commit hash, a unique ID for one saved state of the code, not "latest"), the data snapshot (a frozen copy of the data as of a date) or query with its date, the random seeds (fixed starting numbers that make random steps repeat identically), the environment (pinned library versions, meaning each package fixed at an exact version, recorded in a lock file), and the config for each run, plus the single command that re-runs it. Reason: without this, "it worked for me" cannot be checked, so they spend a week wondering if a different number is a bug or just different data.
2. Results log. A table of every run tried, including the failures: what changed, the metric, and a one-line verdict. Reason: it stops them repeating dead ends.
3. Assumptions and decisions. Each assumption stated as a sentence, marked verified or unverified, with why I chose it (for example "dropped rows with null signup date; about a tenth of rows, roughly 1,200 of 12,000, not checked whether they differ from the rest"). Reason: silent assumptions become silent errors.
4. Open questions and next steps. What is unresolved, ranked by how much it could change the conclusion. This matters most when the goal itself is fuzzy: write down what the stakeholder asked, how I interpreted it, and what I would confirm first. Reason: the successor should not have to rediscover what is unknown.
5. People. Stakeholder, data owner, and me (still reachable), each with the one question they can answer. Reason: ambiguity is cheapest to resolve with the person who owns it.

Worked example (illustrative): half-finished churn model

HANDOVER: churn-model (as of Friday)
Rerun:    git checkout 3f9a2c1; pip install -r requirements.lock
          python train.py --config configs/run_04.yaml   (seed 42)
Data:     churn_events snapshot pulled 2026-09-21, query in sql/pull.sql
Runs:     run_01 baseline logistic (a simple, fast model, the yardstick), run_02 +usage features (better),
          run_03 gradient boosting (a stronger model built from many small decision trees; better, but data leakage: uses a field set after cancellation, DISCARD),
          run_04 fixed leak, current best. Full table: results/runs.csv
Assumed:  churn = no login for 30 days  (UNVERIFIED with Product)
Open:     1) Is 30 days the right churn definition? (biggest risk, changes the label)
          2) Do enterprise accounts behave differently? not yet split
Ask:      Priya (Product) for the definition; Data Eng on-call for snapshot questions

Data leakage means the model was given information that would not exist at prediction time (here, a field only filled in after the customer cancelled), so it looks brilliant in testing and fails in real use. Note run_03: recording a failed run and why it failed is worth more than the good runs, because the leak is exactly what a newcomer would rediscover.

Trade-offs and pitfalls

• A handover longer than two pages does not get read. Keep the top page to these five items and link details.
• Do a cold-start test: ask someone to reproduce run_04 from the note alone, before you leave. If they get stuck, the note is wrong, not them.
• Do not hand over only "the notebook". Notebooks hide execution order and local state.
• Do not present a guess as a finding. Marking "unverified" is what makes the note trustworthy.