The idea in plain words
"Find natural customer groups" is not yet a question a model can answer. Clustering (grouping rows so that rows in a group are similar) will always return groups, whatever you ask of it, but which groups it returns depends on choices the manager has not made: who counts as a customer, what "similar" means, and what the groups are for. Before any algorithm, pin down the decision the groups will feed, because that decides everything else.
What to clarify, and why each answer changes the work
- The decision or action. Targeted marketing, pricing tiers, service levels, inventory planning, or just understanding? Segments for marketing need to be few, named and reachable; segments for exploration can be many. If nobody can name an action that differs by group, there is nothing to cluster for.
- The unit and the time window. One row per customer, per account, per household or per transaction? Over the last 90 days or 3 years? New customers with one purchase and long-standing customers look very different, and a window that is too short mixes seasonal buyers with loyal ones.
- What "similar" should mean. Similar in how much they spend, how often they buy, what they buy, which channel they use, or when they last bought? This is the choice of features, and different choices give different groups. A common starting point for transaction data is RFM: recency (days since last purchase), frequency (number of orders) and monetary value (total spend).
- Who will use the output, and how. A campaign manager needs groups she can describe in a sentence and pull as a list each month. A data feed into a recommender needs stable labels. This sets the number of groups, how often they are refreshed and how they are named.
- Constraints on the groups. A minimum group size to be worth a campaign, groups that must be explainable to a regional team, existing segments that must stay comparable, or fairness or legal limits on using certain attributes.
- Data quality and scope. Refunds and cancellations, test accounts, business buyers mixed with consumers, outliers (one customer buying for a whole company), missing history, customers who merged accounts. These are cheap to ask about now and expensive to discover after the groups look strange.
- How success will be judged. Do groups differ on something the business cares about (repeat rate, margin, response to offers)? Is a simpler rule such as "top 10% by spend" good enough? Agreeing on this before the work also protects you from "that segment doesn't feel right" with no standard to test it against.
Why these questions are not optional: a worked example
Synthetic data: 900 customers from three hidden behaviours (frequent small buyers, rare big buyers, lapsed customers), with spend measured in dollars, orders as counts and recency in days. The features sit on very different scales. The same algorithm (k-means, which picks 3 centre points and repeatedly assigns each customer to the nearest centre and moves each centre to the average of its customers) is run on the raw numbers and on standardised ones (each feature rescaled to mean 0 and standard deviation 1). The adjusted Rand index (ARI, the agreement between two groupings, 1 means identical and about 0 means chance level) scores each against the hidden behaviours. The groups are separated by construction, and the ARI check is possible only because the hidden behaviours are known in synthetic data; on real data there is no answer key, so you judge groups by whether they differ on outcomes the business cares about, are stable across samples and can be described in a sentence. Real data will look less clean.
python
import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score
from sklearn.preprocessing import StandardScaler
rng = np.random.default_rng(0)
n = 900
grp = np.repeat([0, 1, 2], n // 3) # three hidden behaviours: frequent small, rare big, lapsed
orders = np.clip(np.array([20, 4, 3])[grp] + rng.normal(0, 1.5, n), 1, None) # orders in the last year
avg_basket = np.clip(np.array([25, 140, 40])[grp] + rng.normal(0, 8, n), 5, None) # average order value, dollars
days_since = np.clip(np.array([10, 40, 250])[grp] + rng.normal(0, 25, n), 0, None) # days since last purchase
spend = orders * avg_basket # total spend, dollars
X = np.column_stack([spend, orders, days_since])
for name, c in zip(["spend", "orders", "days_since"], X.T):
print(f"{name:<11} min {c.min():7.1f} max {c.max():7.1f}")
def fit(F):
return KMeans(3, n_init=10, random_state=0).fit_predict(F)
raw = fit(X)
scaled = fit(StandardScaler().fit_transform(X))
spend_only = fit(spend.reshape(-1, 1))
print("ARI vs hidden behaviours, raw units: ", round(adjusted_rand_score(grp, raw), 2))
print("ARI vs hidden behaviours, standardised: ", round(adjusted_rand_score(grp, scaled), 2))
print("ARI raw clustering vs spend-only: ", round(adjusted_rand_score(spend_only, raw), 2))
# same customers, a different question: group by what they buy (3 category shares) instead of how
shares = rng.dirichlet(np.ones(3), n)
by_basket = fit(StandardScaler().fit_transform(shares))
print("ARI value-based vs category-based grouping:", round(adjusted_rand_score(scaled, by_basket), 2))
text
spend min 23.4 max 1054.1
orders min 1.0 max 24.6
days_since min 0.0 max 312.2
ARI vs hidden behaviours, raw units: 0.48
ARI vs hidden behaviours, standardised: 1.0
ARI raw clustering vs spend-only: 0.66
ARI value-based vs category-based grouping: 0.0
Two things come out of it.
- The feature and scale choices decide the groups. On raw units the clustering agrees with the hidden behaviours at ARI 0.48, and it agrees with a clustering of spend alone at 0.66: dollars (up to about 1,000) and days (up to about 300) dominate the distance, while the order count (1 to 25) has little weight in it. After standardising, the hidden behaviours are recovered (ARI 1.0). Nothing about the algorithm changed; the preparation did.
- "Similar" is a business choice. Grouping the same customers by value and recency versus by the mix of product categories they buy gives unrelated partitions (ARI 0.0 here, where the category shares are random numbers generated independently of the customers' behaviour, so the zero is built in by construction and shows only that two different definitions of similar can disagree; real columns are usually partly related, so expect something between 0 and 1). Which one is "natural" depends on what the groups are for.
What to say in the room
Restate the request as a question with an owner and an output: "We want groups of customers that differ in purchase value and engagement so the marketing team can run three different retention offers; groups should each be at least 5% of customers and be describable in a sentence. Is that right?" Then propose a simple baseline before any model, for example RFM quintile rules (rank customers on each of recency, frequency and monetary value and cut each ranking into five equal-sized bands, so every customer gets a 1-to-5 score per measure) or spend tiers, so a clustering has to beat something to be adopted.
Common mistakes
- Starting with k-means on whatever columns exist, then looking for a story in the output.
- Assuming clusters are discovered facts. Clustering returns a partition of the data you gave it under the distance you chose; there may be no natural groups at all.
- Skipping the stakeholder's definition of a useful group, so the result is statistically tidy and unusable.
