Logic Regression Solution Boundary

Question

I am implementing logistic regression. I managed to get the probability from this, and I can predict the classification problem of class 2.

My question is:

For my final model, I have weight and training data. There are 2 functions, so my weight is a 2-line vector.

How to do it? I saw this post , but I do not quite understand the answer. Do I need a contour graph?

Answer 1

The advantage of the logistic regression classifier is that as soon as you fit it, you can get the probabilities for any sample vector. This may be an interesting plot. Here is an example using scikit-learn:

import numpy as np from sklearn.linear_model import LogisticRegression from sklearn.datasets import make_classification import matplotlib.pyplot as plt import seaborn as sns sns.set(style="white") 

First create the data and set the classifier in the training set:

 X, y = make_classification(200, 2, 2, 0, weights=[.5, .5], random_state=15) clf = LogisticRegression().fit(X[:100], y[:100]) 

Then make a continuous grid of values ​​and evaluate the probability of each (x, y) point in the grid:

 xx, yy = np.mgrid[-5:5:.01, -5:5:.01] grid = np.c_[xx.ravel(), yy.ravel()] probs = clf.predict_proba(grid)[:, 1].reshape(xx.shape) 

Now draw the probability grid as a contour map and additionally show test case patterns on top of it:

 f, ax = plt.subplots(figsize=(8, 6)) contour = ax.contourf(xx, yy, probs, 25, cmap="RdBu", vmin=0, vmax=1) ax_c = f.colorbar(contour) ax_c.set_label("$P(y = 1)$") ax_c.set_ticks([0, .25, .5, .75, 1]) ax.scatter(X[100:,0], X[100:, 1], c=y[100:], s=50, cmap="RdBu", vmin=-.2, vmax=1.2, edgecolor="white", linewidth=1) ax.set(aspect="equal", xlim=(-5, 5), ylim=(-5, 5), xlabel="$X_1$", ylabel="$X_2$") 

Logistic regression allows you to classify new patterns based on any threshold you want, so it essentially does not have one “decision boundary”. But, of course, the general decision rule is p = .5. We can also simply draw this outline level using the code above:

 f, ax = plt.subplots(figsize=(8, 6)) ax.contour(xx, yy, probs, levels=[.5], cmap="Greys", vmin=0, vmax=.6) ax.scatter(X[100:,0], X[100:, 1], c=y[100:], s=50, cmap="RdBu", vmin=-.2, vmax=1.2, edgecolor="white", linewidth=1) ax.set(aspect="equal", xlim=(-5, 5), ylim=(-5, 5), xlabel="$X_1$", ylabel="$X_2$")