How to do it...

You can create a voting ensemble model for a classification problem using the VotingClassifier class from Python's scikit-learn library. The following steps showcase an example of how to combine the predictions of the decision tree, SVMs, and logistic regression models for a classification problem:

1. Import the required libraries for building the decision tree, SVM, and logistic regression models. We also import VotingClassifier for max-voting:

# Import required libraries
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import VotingClassifier

1. We then move on to building our feature set and creating our train and test datasets:

# We create train & test sample from our dataset
from sklearn.cross_validation import train_test_split

# create feature & response sets
feature_columns = ['sex', 'age', 'Time', 'Number_of_Warts', 'Type', 'Area']
X = df_cryotherapydata[feature_columns]
Y = df_cryotherapydata['Result_of_Treatment']

# Create train & test sets
X_train, X_test, Y_train, Y_test = \
train_test_split(X, Y, test_size=0.20, random_state=1)

1. We build our models with the decision tree, SVM, and logistic regression algorithms:

# create the sub models
estimators = []

dt_model = DecisionTreeClassifier(random_state=1)
estimators.append(('DecisionTree', dt_model))

svm_model = SVC(random_state=1)
estimators.append(('SupportVector', svm_model))

logit_model = LogisticRegression(random_state=1)
estimators.append(('Logistic Regression', logit_model))

1. We build individual models with each of the classifiers we've chosen:

from sklearn.metrics import accuracy_score

for each_estimator in (dt_model, svm_model, logit_model):
    each_estimator.fit(X_train, Y_train)
    Y_pred = each_estimator.predict(X_test)
    print(each_estimator.__class__.__name__, accuracy_score(Y_test, Y_pred))

We can then see the accuracy score of each of the individual base learners:

1. We proceed to ensemble our models and use VotingClassifier to score the accuracy of the ensemble model:

#Using VotingClassifier() to build ensemble model with Hard Voting
ensemble_model = VotingClassifier(estimators=estimators, voting='hard')

ensemble_model.fit(X_train,Y_train)
predicted_labels = ensemble_model.predict(X_test) 

print("Classifier Accuracy using Hard Voting: ", accuracy_score(Y_test, predicted_labels))

We can see the accuracy score of the ensemble model using Hard Voting: