We start this tutorial to exhibit harness TPOT to automate and optimize machine studying pipelines virtually. By working instantly in Google Colab, we make sure the setup is light-weight, reproducible, and accessible. We stroll via loading knowledge, defining a customized scorer, tailoring the search house with superior fashions like XGBoost, and organising a cross-validation technique. As we proceed, we discover how evolutionary algorithms in TPOT seek for high-performing pipelines, offering us transparency via Pareto fronts and checkpoints. Take a look at the FULL CODES right here.
!pip -q set up tpot==0.12.2 xgboost==2.0.3 scikit-learn==1.4.2 graphviz==0.20.3
import os, json, math, time, random, numpy as np, pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split, StratifiedKFold
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import make_scorer, f1_score, classification_report, confusion_matrix
from sklearn.pipeline import Pipeline
from tpot import TPOTClassifier
from sklearn.linear_model import LogisticRegression
from sklearn.naive_bayes import GaussianNB
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier, GradientBoostingClassifier
from xgboost import XGBClassifier
SEED = 7
random.seed(SEED); np.random.seed(SEED); os.environ["PYTHONHASHSEED"]=str(SEED)We start by putting in the libraries and importing all of the important modules that help knowledge dealing with, mannequin constructing, and pipeline optimization. We set a set random seed to make sure our outcomes stay reproducible each time we run the pocket book. Take a look at the FULL CODES right here.
X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.3, stratify=y, random_state=SEED)
scaler = StandardScaler().match(X_tr)
X_tr_s, X_te_s = scaler.rework(X_tr), scaler.rework(X_te)
def f1_cost_sensitive(y_true, y_pred):
return f1_score(y_true, y_pred, common="binary", pos_label=1)
cost_f1 = make_scorer(f1_cost_sensitive, greater_is_better=True)Right here, we load the breast most cancers dataset and cut up it into coaching and testing units whereas preserving class steadiness. We standardize the options for stability after which outline a customized F1-based scorer, permitting us to guage pipelines with a deal with successfully capturing constructive circumstances. Take a look at the FULL CODES right here.
tpot_config = {
'sklearn.linear_model.LogisticRegression': {
'C': [0.01, 0.1, 1.0, 10.0],
'penalty': ['l2'], 'solver': ['lbfgs'], 'max_iter': [200]
},
'sklearn.naive_bayes.GaussianNB': {},
'sklearn.tree.DecisionTreeClassifier': {
'criterion': ['gini','entropy'], 'max_depth': [3,5,8,None],
'min_samples_split':[2,5,10], 'min_samples_leaf':[1,2,4]
},
'sklearn.ensemble.RandomForestClassifier': {
'n_estimators':[100,300], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.ExtraTreesClassifier': {
'n_estimators':[200], 'criterion':['gini','entropy'],
'max_depth':[None,8], 'min_samples_split':[2,5], 'min_samples_leaf':[1,2]
},
'sklearn.ensemble.GradientBoostingClassifier': {
'n_estimators':[100,200], 'learning_rate':[0.03,0.1],
'max_depth':[2,3], 'subsample':[0.8,1.0]
},
'xgboost.XGBClassifier': {
'n_estimators':[200,400], 'max_depth':[3,5], 'learning_rate':[0.05,0.1],
'subsample':[0.8,1.0], 'colsample_bytree':[0.8,1.0],
'reg_lambda':[1.0,2.0], 'min_child_weight':[1,3],
'n_jobs':[0], 'tree_method':['hist'], 'eval_metric':['logloss'],
'gamma':[0,1]
}
}
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=SEED)We outline a customized TPOT configuration that mixes linear fashions, tree-based learners, ensembles, and XGBoost, using fastidiously chosen hyperparameters. We additionally established a stratified 5-fold cross-validation technique, guaranteeing that each candidate pipeline is examined pretty throughout balanced splits of the dataset. Take a look at the FULL CODES right here.
t0 = time.time()
tpot = TPOTClassifier(
generations=5,
population_size=40,
offspring_size=40,
scoring=cost_f1,
cv=cv,
subsample=0.8,
n_jobs=-1,
config_dict=tpot_config,
verbosity=2,
random_state=SEED,
max_time_mins=10,
early_stop=3,
periodic_checkpoint_folder="tpot_ckpt",
warm_start=False
)
tpot.match(X_tr_s, y_tr)
print(f"n⏱️ First search took {time.time()-t0:.1f}s")
def pareto_table(tpot_obj, ok=5):
rows=[]
for ind, meta in tpot_obj.pareto_front_fitted_pipelines_.gadgets():
rows.append({
"pipeline": ind, "cv_score": meta['internal_cv_score'],
"dimension": len(str(meta['pipeline'])),
})
df = pd.DataFrame(rows).sort_values("cv_score", ascending=False).head(ok)
return df.reset_index(drop=True)
pareto_df = pareto_table(tpot, ok=5)
print("nTop Pareto pipelines (cv):n", pareto_df)
def eval_pipeline(pipeline, X_te, y_te, title):
y_hat = pipeline.predict(X_te)
f1 = f1_score(y_te, y_hat)
print(f"n[{name}] F1(take a look at) = {f1:.4f}")
print(classification_report(y_te, y_hat, digits=3))
print("nEvaluating high pipelines on take a look at:")
for i, (ind, meta) in enumerate(sorted(
tpot.pareto_front_fitted_pipelines_.gadgets(),
key=lambda kv: kv[1]['internal_cv_score'], reverse=True)[:3], 1):
eval_pipeline(meta['pipeline'], X_te_s, y_te, title=f"Pareto#{i}")We launch an evolutionary search with TPOT, cap the runtime for practicality, and checkpoint progress, permitting us to reproducibly hunt for sturdy pipelines. We then examine the Pareto entrance to establish the highest trade-offs, convert it right into a compact desk, and choose leaders based mostly on the cross-validation rating. Lastly, we consider the most effective candidates on the held-out take a look at set to substantiate real-world efficiency with F1 and a full classification report. Take a look at the FULL CODES right here.
print("n🔁 Heat-start for additional refinement...")
t1 = time.time()
tpot2 = TPOTClassifier(
generations=3, population_size=40, offspring_size=40,
scoring=cost_f1, cv=cv, subsample=0.8, n_jobs=-1,
config_dict=tpot_config, verbosity=2, random_state=SEED,
warm_start=True, periodic_checkpoint_folder="tpot_ckpt"
)
strive:
tpot2._population = tpot._population
tpot2._pareto_front = tpot._pareto_front
besides Exception:
cross
tpot2.match(X_tr_s, y_tr)
print(f"⏱️ Heat-start additional search took {time.time()-t1:.1f}s")
best_model = tpot2.fitted_pipeline_ if hasattr(tpot2, "fitted_pipeline_") else tpot.fitted_pipeline_
eval_pipeline(best_model, X_te_s, y_te, title="BestAfterWarmStart")
export_path = "tpot_best_pipeline.py"
(tpot2 if hasattr(tpot2, "fitted_pipeline_") else tpot).export(export_path)
print(f"n📦 Exported finest pipeline to: {export_path}")
from importlib import util as _util
spec = _util.spec_from_file_location("tpot_best", export_path)
tbest = _util.module_from_spec(spec); spec.loader.exec_module(tbest)
reloaded_clf = tbest.exported_pipeline_
pipe = Pipeline([("scaler", scaler), ("model", reloaded_clf)])
pipe.match(X_tr, y_tr)
eval_pipeline(pipe, X_te, y_te, title="ReloadedExportedPipeline")
report = {
"dataset": "sklearn breast_cancer",
"train_size": int(X_tr.form[0]), "test_size": int(X_te.form[0]),
"cv": "StratifiedKFold(5)",
"scorer": "customized F1 (binary)",
"search": {"gen_1": 5, "gen_2_warm": 3, "pop": 40, "subsample": 0.8},
"exported_pipeline_first_120_chars": str(reloaded_clf)[:120]+"...",
}
print("n🧾 Mannequin Card:n", json.dumps(report, indent=2))We proceed the search with a heat begin, reusing the discovered heat begin to refine candidates and choose the most effective performer on our take a look at set. We export the successful pipeline, reload it alongside our scaler to imitate deployment, and confirm its outcomes. Lastly, we generate a compact mannequin card to doc the dataset, search settings, and the abstract of the exported pipeline for reproducibility.
In conclusion, we see how TPOT permits us to maneuver past trial-and-error mannequin choice and as an alternative depend on automated, reproducible, and explainable optimization. We export the most effective pipeline, validate it on unseen knowledge, and even reload it for deployment-style use, confirming that the workflow isn’t just experimental however production-ready. By combining reproducibility, flexibility, and interpretability, we finish with a sturdy framework that we will confidently apply to extra advanced datasets and real-world issues.
Take a look at the FULL CODES right here. Be happy to take a look at our GitHub Web page for Tutorials, Codes and Notebooks. Additionally, be at liberty to observe us on Twitter and don’t overlook to hitch our 100k+ ML SubReddit and Subscribe to our Publication.
Asif Razzaq is the CEO of Marktechpost Media Inc.. As a visionary entrepreneur and engineer, Asif is dedicated to harnessing the potential of Synthetic Intelligence for social good. His most up-to-date endeavor is the launch of an Synthetic Intelligence Media Platform, Marktechpost, which stands out for its in-depth protection of machine studying and deep studying information that’s each technically sound and simply comprehensible by a large viewers. The platform boasts of over 2 million month-to-month views, illustrating its recognition amongst audiences.
