Marketplace Dynamics and Multi-Sided Platforms Questions

Framework: treat this as intermittent-count forecasting + probabilistic demand aggregation for safety stock. Steps below.

1. Data & preprocessing

• Aggregate to daily SKU-level. Create binary demand flag, non-zero size, streak lengths, recency features (days-since-last-sale), calendar (weekday/holiday), price/promotions, inventory/lead-time features.
• Label demand regimes (sporadic, intermittent, lumpy) using Croston/Booth criteria (d/nz ratio).
• Impute missing data, remove obvious anomalies, cap outliers in sale size.

2. Baseline model families

• Croston / Syntetos–Boylan (SBC) / TSB: simple decomposition of demand size and inter-arrival times for intermittent series.
• Count models: ZIP / ZINB for zero-inflation and overdispersion; incorporate covariates with GLM/GLMM.
• Hierarchical Bayesian: partial pooling across SKUs to borrow strength—model zero-inflation and purchase size as linked hierarchical components with SKU-group priors.
• ML hybrids: tree ensembles or CatBoost/XGBoost predicting two-stage (P(demand>0) and conditional demand size) or mixture density outputs. Use features above plus embeddings for SKU/category.
• Bootstrapped nonparametric: residual/block bootstrap for series with weak parametric fit.

3. Cross-SKU pooling strategies

• Cluster SKUs by demand profile, category, lead time, or lifecycle stage.
• Use hierarchical models (random effects by SKU/category) for partial pooling to stabilize parameter estimates for low-sales SKUs.
• Meta-model: for very sparse SKUs, use cluster-level models; for richer SKUs, use SKU-specific hybrids.

4. Probabilistic forecasts & safety stock

• Produce full predictive distribution per day via: parametric (ZIP/ZINB predictive), Bayesian posterior predictive sampling, or quantile regression / bootstrapping of residuals.
• Aggregate lead-time demand by convolution (simulate daily draws for lead time horizon) to get mean, variance, and quantiles.
• Map to safety stock: choose service level α and compute SS = quantile_LT(α) - mean_LT (or use z * std_LT if normality approx); compare fill-rate vs. cycle-service level metrics.

5. Validation & metrics

• Use rolling time-series CV respecting chronology. Evaluate CRPS, quantile coverage, bias, RMSLE, and service-level-oriented metrics (stockouts, expected lost sales).
• Back-test safety-stock policy with simulated inventory policy (s, S or reorder point) and compute service level vs. cost trade-offs.

6. Production considerations

• Automation pipeline for feature refresh, model selection per SKU cluster, monitoring drift (calibration of probabilities), and periodic Bayesian/ensemble retraining.
• Explainability: expose P(d>0) and expected demand components to ops stakeholders.

Trade-offs: Croston family is simple and fast for many SKUs; hierarchical Bayesian gives best uncertainty for sparse data but is heavier compute. Hybrid two-stage models often hit the sweet spot: robust P(>0) from trees + parametric/count model for size, combined into probabilistic forecasts for safety-stock calculation.

Two-sided network effects occur when value to one side of a marketplace (e.g., buyers) increases as the other side (e.g., sellers) grows, and vice versa — more sellers attract more buyers, creating positive feedback.

How liquidity, matching quality, and pricing incentives interact:

• Liquidity (supply and demand depth) is the foundation: high liquidity reduces wait time and increases likelihood of successful transactions, amplifying network effects.
• Matching quality (relevance, accuracy) converts liquidity into perceived value; poor matching means more participants but low satisfaction, weakening effects.
• Pricing incentives (fees, subsidies, promotions) shape participation and behavior: well-designed incentives bootstrap two sides and sustain activity, while mispriced incentives can attract low-quality supply or chase cheap demand that leaves once subsidies stop.

Three metrics to detect weakening network effects:

1. Fill rate / conversion rate (requests matched to completed transactions) — falling fill rate signals liquidity or matching issues.
2. Time-to-match / wait time — increasing wait times indicate thinning opposite-side supply/demand.
3. Retention or repeat-usage rate by cohort — declining retention suggests reduced perceived value.

Mitigation strategy:
Run targeted experiments to improve matching and liquidity: use dynamic pricing/priority matching for underserved segments (e.g., subsidize high-quality suppliers in regions with long wait times), A/B test matching algorithms (incorporating richer features or ML ranking), and monitor the above metrics by cohort to iterate quickly.

Approach: Use importance sampling to reweight logged rewards by ratio of target-policy probability to logged propensity (IPS). Doubly-Robust (DR) adds a reward model to reduce variance and bias: DR = estimated reward under target + weighted residuals.

# pseudocode (python-like)
# data: list of records (context x, action a, propensity p_log, reward r)
# target_policy: function pi(a | x)
# reward_model: function mu_hat(x, a) predicting E[r | x,a]  (for DR)

def IPS(data, target_policy, clip=None, self_normalize=False):
    weights = []
    vals = []
    for x,a,p_log,r in data:
        p_target = target_policy.prob(a, x)
        w = p_target / p_log
        if clip is not None:
            w = min(w, clip)
        weights.append(w)
        vals.append(w * r)
    if self_normalize:
        Z = sum(weights)
        return sum(vals) / Z if Z>0 else 0.0
    return sum(vals) / len(data)

def DR(data, target_policy, reward_model, clip=None, self_normalize=False):
    weights = []
    residuals = []
    baseline = []
    for x,a,p_log,r in data:
        p_target = target_policy.prob(a, x)
        w = p_target / p_log
        if clip is not None:
            w = min(w, clip)
        mu_target = sum(target_policy.prob(a2,x)*reward_model.predict(x,a2) for a2 in target_policy.actions(x))
        mu_hat = reward_model.predict(x,a)
        weights.append(w)
        residuals.append(w * (r - mu_hat))
        baseline.append(mu_target)
    if self_normalize:
        Z = sum(weights)
        return (sum(baseline) + sum(residuals)) / Z if Z>0 else 0.0
    return (sum(baseline) + sum(residuals)) / len(data)

Key points:

• IPS unbiased when propensities correct; high variance when p_log small.
• DR is consistent if either propensities or reward model is correct; often lower variance.

Handling high-variance weights:

• Clipping (cap w at tau) trades bias for variance reduction.
• Self-normalized IS divides weighted sum by sum of weights to stabilize scale.
• Use stabilized weights: w' = w / (w + lambda) or use weight trimming based on quantiles.
• Bootstrap or robust CIs for uncertainty.

Non-stationary propensities:

• Re-estimate logging propensities per time window; include time as context feature.
• Use time-decayed weighting or importance ratios adjusted for policy-change timestamps.
• If propensities unknown or misrecorded, fit a behavior policy model (probability of action|x,t) and use its predictions in place of p_log.
• Validate with diagnostics: effective sample size ESS = (sum w)^2 / sum(w^2), check for low ESS and stratify evaluation.

Edge cases & practical tips:

• Ensure p_log > 0 for actions with nonzero p_target; otherwise restrict target to supported actions.
• Regularize reward model, cross-validate, and inspect residuals for bias.
• Report both IPS and DR estimates and their CIs; perform sensitivity analyses across clipping thresholds.

Situation: We need a multi-echelon mixed-integer program (MIP) that chooses production, shipments, inventory and stockout decisions across factories (F), distribution centers (D) and retail nodes (R) to minimize expected total cost while meeting service level, capacity and flow conservation constraints.

Decision variables:

• Continuous:
  • x_f,t ≥ 0: production at factory f in period t
  • y_{i,j,t} ≥ 0: shipment quantity from node i to j in period t (i∈F∪D, j∈D∪R)
  • I_{n,t} ≥ 0: on-hand inventory at node n (n∈F∪D∪R) at end of t
  • S_{r,t} ≥ 0: lost sales or backorders at retail r in t
• Binary:
  • z_{i,j,t} ∈ {0,1}: open/activate shipment route i→j in t (if fixed cost or limited routes)

Parameters:

• c^p_f: unit production cost; c^{tr}_{i,j}: unit transport cost; h_n: unit holding cost; c^s_r: stockout cost per unit (or penalty)
• Cap^p_f, Cap^{tr}_{i,j}, Cap^inv_n: capacities
• D_{r,t}: demand (stochastic) at retail r in t; P scenarios ω with probability p_ω if stochastic

Objective (expected total cost over scenarios):
Minimize sum_t [ Σ_f c^p_f x_{f,t} + Σ_{i,j} c^{tr}{i,j} y{i,j,t} + Σ_n h_n I_{n,t} + Σ_r c^s_r S_{r,t} + Σ_{i,j} K_{i,j} z_{i,j,t} ] averaged over scenarios (or weighted by p_ω).

Core constraints:

1. Flow conservation (for each non-factory node n and scenario ω, period t):
   I_{n,t-1} + Σ_{i} y_{i,n,t} + (inflow from production if factory) - Σ_{j} y_{n,j,t} = D_{n,t}^ω - S_{n,t} + I_{n,t}
   For retail, consumption equals demand minus stockout.
2. Production & transport capacity:
   x_{f,t} ≤ Cap^p_f
   y_{i,j,t} ≤ Cap^{tr}{i,j} * z{i,j,t}
3. Inventory nonnegativity and linking:
   I_{n,t} ≥ 0; S_{r,t} ≥ 0
4. Service level constraint (chance or equivalent):
   Chance form: Pω( S_{r,t} ≤ s_threshold ) ≥ 1 - α OR
   Deterministic equivalent using safety stock: I_{r,t} ≥ Φ^{-1}(1-α) * σ_{demand,r,t} + expected lead-time demand
   Or use Expected Shortfall constraint: Eω[S_{r,t}] ≤ allowable_stockout
5. Integrality if route decisions or setup decisions required.

Stochastic modeling:

• Use scenario-based stochastic programming with first-stage decisions (e.g., production plans, route openings) and second-stage recourse (shipments, inventories, stockouts) per scenario. Solve expected cost over scenarios.

Scalability & heuristics for large instances:

• Scenario reduction or Sample Average Approximation (SAA) to approximate expectation with manageable scenarios.
• Decomposition:
  • Benders decomposition (master: integer first-stage vars z, x; subproblems: scenario-specific linear recourse solved in parallel; generate Benders cuts).
  • Lagrangian relaxation: relax coupling capacity or flow constraints; solve separable subproblems per facility or scenario; update multipliers via subgradient.
  • Dantzig–Wolfe / column generation for routing/lot-sizing formulations.
• Heuristics and metaheuristics:
  • Rolling-horizon heuristic: solve shorter horizons sequentially with warm starts.
  • Greedy + local search: fix route openings greedily, then optimize continuous variables; apply local search (swap open/closed routes).
  • Hybrid matheuristics: combine MILP for aggregate planning and heuristics for allocation; use LP-relaxation rounding.
• Practical tips:
  • Tighten formulation with valid inequalities (cover constraints, flow bounds).
  • Preprocess capacities/demand to prune infeasible arcs.
  • Use warm starts from LP-relaxation and exploit parallel solvers for scenario subproblems.
    Expected trade-offs: exact MIP gives optimality but scales poorly with scenarios and network size; decomposition and heuristics trade optimality gap for tractability and speed—choose based on service-level risk tolerance and compute budget.

This model can be extended to multi-period lead times, perishability, and dynamic pricing; calibrate cost/penalty parameters to reflect business KPIs (fill rate, lost-sales vs backorder).

We want a partial-pooling hierarchical model for log requests (or log demand) by city to estimate price elasticity while sharing strength across cities.

Model sketch

• y_{c,t} = log(requests_{c,t})
• x_{c,t} = log(price_{c,t})
• Include city-level intercepts and price slopes (elasticities), time fixed effects, and short-term shocks.

Statistical model
y_{c,t} ~ Normal(mu_{c,t}, sigma_y)
mu_{c,t} = alpha_c + beta_c * x_{c,t} + gamma_t + z_{c,t}
alpha_c ~ Normal(alpha_mu, alpha_sigma)
beta_c ~ Normal(beta_mu, beta_sigma) # elasticity per city (partial pooling)
(alpha_mu, beta_mu) ~ Normal(0, 5)
(alpha_sigma, beta_sigma) ~ HalfNormal(2)
gamma_t ~ Normal(0, sigma_gamma) # time fixed effects (daily/week)
sigma_gamma ~ HalfNormal(1)
z_{c,t} ~ AR1(phi, sigma_z) # short-term autocorrelated shocks per city
phi ~ Beta(2,2)
sigma_z ~ HalfNormal(1)
sigma_y ~ HalfNormal(1)

Implementation (PyMC3 style)

import pymc3 as pm
with pm.Model() as model:
    # Hyperpriors
    alpha_mu = pm.Normal('alpha_mu', 0, 5)
    beta_mu  = pm.Normal('beta_mu', 0, 5)
    alpha_sigma = pm.HalfNormal('alpha_sigma', 2)
    beta_sigma  = pm.HalfNormal('beta_sigma', 2)

    # City effects
    alpha = pm.Normal('alpha', alpha_mu, alpha_sigma, shape=C)
    beta  = pm.Normal('beta', beta_mu, beta_sigma, shape=C)

    # Time fixed effects
    gamma = pm.Normal('gamma', 0, 1, shape=T)

    # AR(1) shocks per city
    phi = pm.Beta('phi', 2, 2)
    sigma_z = pm.HalfNormal('sigma_z', 1)
    z = pm.AR('z', rho=phi, sigma=sigma_z, shape=(C,T))

    sigma_y = pm.HalfNormal('sigma_y', 1)
    mu = alpha[city_idx] + beta[city_idx]*x_log + gamma[time_idx] + z[city_idx, time_idx]
    y = pm.Normal('y', mu, sigma_y, observed=y_log)

Inference

• Prefer NUTS MCMC for accurate posteriors (run multiple chains, check R-hat < 1.05, ESS).
• For very large data consider ADVI/ADVI+NUTS warmstart or minibatch variational inference; validate VI via posterior predictive checks.

Credible intervals & decision-making

• Extract posterior samples for beta_c to compute 95% credible intervals (percentile or highest posterior density).
• For pricing decisions, simulate counterfactuals: sample posterior, compute predicted demand and revenue at candidate prices, produce distribution of revenue lift; choose price maximizing expected revenue or constrained quantile (e.g., 5th percentile to be conservative).

Posterior predictive checks

• Generate y_rep from the posterior predictive and compare:
  • Time series plots (observed vs median y_rep with credible bands) per city
  • Test statistics: mean, variance, autocorrelation, and tail behavior
  • Probability integral transform / calibration plots
• If systematic misfit, iterate: add covariates (promotions, holidays), allow nonlinearity (splines, price^2), or relax AR(1).

Practical notes

• Center/scale x_log and include priors informed by domain (elasticities typically between -0.1 and -3).
• Use informative priors if data sparse in some cities to stabilize estimates.
• Report city-level posterior summaries, pooled mean elasticity, and decision-scenario posterior predictive distributions to stakeholders.