Duration is explicitly forbidden
The feature becomes available only after contact. A test fails if it reaches the training matrix, keeping offline quality honest.
Public data · Independently developed · Reproducible
Campaign Decisioning Lab
A production-shaped machine-learning pipeline that answers a practical marketing question: under a finite contact budget, who should the team prioritize—and how much confidence should it place in that ranking?
✓ Clean-room implementation
✓ UCI CC BY 4.0 data
✓ Measured metrics published
No live endpoint claimed
DecisionRank outreach under a fixed budget
Core stackpandas · NumPy · scikit-learn · SQL
EvidenceOrdered holdout · bootstrap · tests · model card
SourceView repository ↗
01 / Problem
A probability is useful only when it changes a decision.
The public UCI Bank Marketing dataset records direct-marketing calls and whether each customer subscribed to a term deposit. A conventional tutorial would optimize accuracy and stop. This project treats the model as one component in a contact policy: rank candidates, select a budget, expose the value assumptions, and show the trade-off between reach and precision.
The primary user is an analytics or campaign team choosing the next audience tranche. The output is a scored and queryable decision table—not a screenshot of a notebook.
Why the naive approach fails
Class imbalance makes accuracy uninformative. A default 0.5 threshold ignores capacity and economics. Most importantly, call duration is only known after the call and would leak future information into a pre-contact targeting model.
02 / System
One command from authoritative data to an auditable decision table.
SourceUCI archivedownload · checksum · license
ValidateSchema & leakage gatetypes · target · forbidden fields
LearnBaseline + candidatesordered split · preprocessing
EvaluateQuality + policyPR-AUC · Brier · lift · bootstrap
ServeSQLite decision martscores · segments · ranked SQL
The Python package keeps ingestion, feature policy, modeling, evaluation, segmentation, and SQL export separate. That separation makes each layer testable and lets a future batch deployment replace local storage without changing the analytical contract.
03 / Design
Controls that matter more than squeezing out another decimal.
Order-preserving holdout
The source data is ordered chronologically. Earlier observations train the models; later observations validate and test them.
Rank first, threshold second
Scores remain separate from contact capacity and value assumptions. Teams can inspect policy outcomes at multiple budget levels.
Segments describe, they do not decide
PCA and K-Means create diagnostic audience profiles. Unsupervised labels never override predicted response or human policy.
Bootstrap intervals
Key metrics include sampling uncertainty so a small apparent gain is not presented as certainty.
Activation-ready output
Scored rows are written to SQLite and ranked with window functions, demonstrating a handoff beyond Python.
04 / Results
Measured on a later ordered holdout—with the selection history disclosed.
The default 70/15/15 split selected logistic regression on 6,178 validation rows, then measured it on 6,179 later rows. These values load from a reduced pipeline artifact rather than hand-entered claims. Because the split was refined during sensitivity analysis, the final window is useful engineering evidence—not a pristine, prospectively locked confirmatory estimate.
PR-AUCPipelineimbalance-aware discrimination
ROC-AUCPipelineranking across thresholds
Brier scorePipelineprobability error · lower is better
Capture @ 20%Pipelinepositives in top audience tranche
Stress result kept visible
Under an earlier 60/20/20 cutoff, logistic regression reached 0.0972 validation PR-AUC versus 0.1107 for the constant-prior baseline, so the correct decision was to reject both trained challengers. The regime sensitivity is a monitoring risk, not a result to hide.
Interpretation boundary
This is propensity modeling, not causal uplift. The system identifies customers whose observed patterns resemble prior responders; it does not claim the call itself caused conversion. A randomized-treatment dataset is the correct next step for incremental-effect estimation.
05 / Reproduce
Readable locally, testable in CI, portable to batch infrastructure.
python -m venv .venv
.venv/Scripts/activate
python -m pip install -e "projects/campaign-intelligence[dev]"
campaign-intelligence run --data-dir data --artifacts-dir artifacts \
--contact-cost 1 --conversion-value 10 --contact-fraction 0.20
pytest projects/campaign-intelligence/tests
Repository evidence
- Packaged source under
src/, typed boundaries, deterministic seeds, and a CLI. - Unit tests for schema, leakage, splitting, metrics, serialization, and SQL output.
- Data card, model card, assumptions, limitations, and explicit dataset attribution.
- Docker packaging and GitHub Actions; generated data and model binaries stay out of Git.
Operational next step
The natural AWS path is scheduled batch scoring: source objects in S3, a containerized processing job, versioned artifacts, a data-quality gate, and CloudWatch alarms for schema and score-distribution drift. This repository does not claim that deployment until an AWS account produces verifiable run evidence.
Next case study