What should I use instead of bootstrap?

Question

When I run this code:

from sklearn import cross_validation bs = cross_validation.Bootstrap(9, random_state=0) 

I get this failure warning:

 C:\Anaconda\envs\p33\lib\site-packages\sklearn\cross_validation.py:684: DeprecationWarning: Bootstrap will no longer be supported as a cross-validation method as of version 0.15 and will be removed in 0.17 "will be removed in 0.17", DeprecationWarning) 

What should be used instead of bootstrap?

Answer 1

From the scikit-learn 0.15 release notes in the API Changes Summary

• cross_validation.Bootstrap deprecated. cross_validation.KFold or cross_validation.ShuffleSplit .

And from the source code itself :

 # See, eg, http://youtu.be/BzHz0J9a6k0?t=9m38s for a motivation # behind this deprecation warnings.warn("Bootstrap will no longer be supported as a " + "cross-validation method as of version 0.15 and " + "will be removed in 0.17", DeprecationWarning) 

Answer 2

You can use BaggingClassifier :

 bag = BaggingClassifier(base_estimator=your_estimator, n_estimators=100, max_samples=1.0, bootstrap=True, n_jobs=-1) bag.fit(X, y) recalls = [] for estimator, samples in zip(bag.estimators_, bag.estimators_samples_): # compute predictions on out-of-bag samples mask = ~samples y_pred = estimator.predict(X[mask]) # compute some statistic recalls.append(recall(y[mask], y_pred)) # Do something with stats, eg find confidence interval print(np.percentile(recalls, [2.5, 97.5]))