Use PyCaret to Build Classification Machine Learning Model

 Use PyCaret to Build Classification Machine Learning Model

My first machine learning model in Python for a hackathon was quite a cumbersome block of code. I still remember the many lines of code it took to build an ensemble model – it would have taken a wizard to untangle that mess!

When it comes to building interpretable machine learning models, especially in the industry (or when we want to explain our hackathon results to the client), writing efficient code is key to success. That’s why I strongly recommend using the PyCaret library.

I wish PyCaret was around during my rookie machine learning days! It is a super flexible and useful library that I’ve leaned on quite a bit in recent months. I firmly believe anyone with an aspiration to succeed as a data science or analytics professional will benefit a lot from using PyCaret.

We’ll see what exactly PyCaret it, how to install it on your machine, and then we’ll dive into using PyCaret for building interpretable machine learning models, including ensemble models. A lot of learning to be done so let’s dig in.

 

Use PyCaret to Build Classification Machine Learning Model

Table of Contents

1. What is PyCaret and Why Should you Use it?
2. Installing PyCaret on your Machine
3. Let’s Get Familiar with PyCaret
4. Training our Machine Learning Model using PyCaret
5. Building Ensemble Models using PyCaret
6. Let’s Analyze our Model!
7. Time to Make Predictions
8. Save and Load the Model

 

What is PyCaret and Why Should you Use it?

PyCaret is an open-source, machine learning library in Python that helps you from data preparation to model deployment. It is easy to use and you can do almost every data science project task with just one line of code.

I’ve found PyCaret extremely handy. Here are two primary reasons why:

• PyCaret, being a low-code library, makes you more productive. You can spend less time on coding and can do more experiments
• It is an easy to use machine learning library that will help you perform end-to-end machine learning experiments, whether that’s imputing missing values, encoding categorical data, feature engineering, hyperparameter tuning, or building ensemble models

 

Installing PyCaret on your Machine

This is as straightforward as it gets. You can install the first stable version of PyCaret, v1.0.0, directly using pip. Just run the below command in your Jupyter Notebook to get started:

!pip3 install pycaret

 

Let’s Get Familiar with PyCaret

Problem Statement and Dataset

In this article, we are going to solve a classification problem. We have a bank dataset with features like customer age, experience, income, education, and whether he/she has a credit card or not. The bank wants to build a machine learning model that will help them identify the potential customers who have a higher probability of purchasing a personal loan.

The dataset has 5000 rows and we have kept 4000 for training our model and the remaining 1000 for testing the model. You can find the complete code and dataset used in this article here.

Let’s start by reading the dataset using the Pandas library:

	# importing pandas to read the CSV file
	import pandas as pd
	# read the data
	data_classification = pd.read_csv('datasets/loan_train_data.csv')
	# view the top rows of the data
	data_classification.head()

view rawpycaret_1.py hosted with  by GitHub

The very first step before we start our machine learning project in PyCaret is to set up the environment. It’s just a two-step process:

1. Importing a Module: Depending upon the type of problem you are going to solve, you first need to import the module. In the first version of PyCaret, 6 different modules are available – regression, classification, clustering, natural language processing (NLP), anomaly detection, and associate mining rule. In this article, we will solve a classification problem and hence we will import the classification module
2. Initializing the Setup: In this step, PyCaret performs some basic preprocessing tasks, like ignoring the IDs and Date Columns, imputing the missing values, encoding the categorical variables, and splitting the dataset into the train-test split for the rest of the modeling steps. When you run the setup function, it will first confirm the data types, and then if you press enter, it will create the environment for you to go ahead

	# import the classification module
	from pycaret import classification
	# setup the environment
	classification_setup = classification.setup(data= data_classification, target='Personal Loan')

view rawpycaret_2.py hosted with  by GitHub

We’re all set to explore PyCaret!

 

Training our Machine Learning Model using PyCaret

Training a Model

Training a model in PyCaret is quite simple. You just need to use the create_model function that takes just the one parameter – the model abbreviation as a string. Here, we are going to first train a decision tree model for which we have to pass “dt” and it will return a table with k-fold cross-validated scores of common evaluation metrics used for classification models.

Here’s q quick reminder of the evaluation metrics used for supervised learning:

• Classification: Accuracy, AUC, Recall, Precision, F1, Kappa
• Regression: MAE, MSE, RMSE, R2, RMSLE, MAPE

You can check the documentation page of PyCaret for more abbreviations.

	# build the decision tree model
	classification_dt = classification.create_model('dt')

view rawpycaret_3.py hosted with  by GitHub

Similarly, for training the XGBoost model, you just need to pass the string “xgboost“:

	# build the xgboost model
	classification_xgb = classification.create_model('xgboost')

view rawpycaret_4.py hosted with  by GitHub

 

Hyperparameter Tuning

We can tune the hyperparameters of a machine learning model by just using the tune_model function which takes one parameter – the model abbreviation string (the same as we used in the create_model function).

PyCaret provides us a lot of flexibility. For example, we can define the number of folds using the fold parameter within the tune_model function. Or we can change the number of iterations using the n_iter parameter. Increasing the n_iter parameter will obviously increase the training time but will give a much better performance.

Let’s train a tuned CatBoost model:

	# build and tune the catboost model
	tune_catboost = classification.tune_model('catboost')

view rawpycaret_5.py hosted with  by GitHub

 

Building Ensemble Models using PyCaret

Ensemble models in machine learning combine the decisions from multiple models to improve the overall performance.

In PyCaret, we can create bagging, boosting, blending, and stacking ensemble models with just one line of code.

If you want to learn ensemble models in-depth, I would highly recommend this article: A Comprehensive Guide to Ensemble Learning.

Let’s train a boosting ensemble model here. It will also return a table with k-fold cross-validated scores of common evaluation metrics:

	# ensemble boosting
	boosting = classification.ensemble_model(classification_dt, method= 'Boosting')

view rawpycaret_boosting.py hosted with  by GitHub

Another very famous ensembling technique is blending. You just need to pass the models that you have created in a list of the blend_models function.

That’s it! You just need to write a single line of code in PyCaret to do most of the stuff.

	# Ensemble: blending
	blender = classification.blend_models(estimator_list=[classification_dt, classification_xgb])

view rawpycaret_blender.py hosted with  by GitHub

 

Compare Models

This is another useful function of the PyCaret library. If you do not want to try the different models one by one, you can use the compare models function and it will train and compare common evaluation metrics for all the available models in the library of the module you have imported.

This function is only available in the pycaret.classification and pycaret.regression modules.

	# compare performance of different classification models
	classification.compare_models()

view rawcomapre_models.py hosted with  by GitHub

 

Let’s Analyze our Model!

Now, after training the model, the next step is to analyze the results. This especially useful from a business perspective, right? Analyzing a model in PyCaret is again very simple. Just a single line of code and you can do the following:

1. Plot Model Results: Analyzing model performance in PyCaret is as simple as writing plot_model. You can plot decision boundaries, precision-recall curve, validation curve, residual plots, etc.. Also, for clustering models, you can plot the elbow plot and silhouette plot. For text data, you can plot word clouds, bigram and trigram frequency plots, etc.
2. Interpret Results: Interpreting model results helps in debugging the model by analyzing the important features. This is a crucial step in industry-grade machine learning projects. In PyCaret, we can interpret the model by SHAP values and correlation plot with just one line of code (getting to be quite a theme this, isn’t it?)

 

Plot Model Results

You can plot model results by providing the model object as the parameter and the type of plot you want. Let’s plot the AUC-ROC curve and decision boundary:

	# AUC-ROC plot
	classification.plot_model(classification_dt, plot = 'auc')
	# Decision Boundary
	classification.plot_model(classification_dt, plot = 'boundary')

view rawplot_1.py hosted with  by GitHub

Let’s plot the precision-recall curve and validation curve of the trained model:

	# Precision Recall Curve
	classification.plot_model(classification_dt, plot = 'pr')
	# Validation Curve
	classification.plot_model(classification_dt, plot = 'vc')

view rawplot_2.py hosted with  by GitHub

 

Evaluate our Model

If you do not want to plot all these visualizations individually, then the PyCaret library has another amazing function – evaluate_model. In this function, you just need to pass the model object and PyCaret will create an interactive window for you to see and analyze the model in all the possible ways:

	# evaluate model
	classification.evaluate_model(classification_dt)

view rawevaluate_model.py hosted with  by GitHub

 

 

 

Pretty cool!

 

Interpret our Model

Interpreting complex models is very important in most machine learning projects. It helps in debugging the model by analyzing what the model thinks is important. In PyCaret, this step is as simple as writing interpret_model to get the Shapley values.

You can read about the Shapley Values here: A Unique Method for Machine Learning Interpretability: Game Theory & Shapley Values.

	# interpret_model: SHAP
	classification.interpret_model(classification_xgb)

view rawinterpret_model.py hosted with  by GitHub

Let’s try to plot the correlation plot:

	# interpret model : Correlation
	classification.interpret_model(classification_xgb,plot='correlation')

view rawinterpret_model_2.py hosted with  by GitHub

 

 

Time to Make Predictions!

Finally, we will make predictions on unseen data. For this, we just need to pass the model that we will use for the predictions and the dataset. Make sure it is in the same format as we provided while setting up the environment earlier. PyCaret builds a pipeline of all the steps and will pass the unseen data into the pipeline and give us the results.

Let’s see how to predict the labels on unseen data:

	# read the test data
	test_data_classification = pd.read_csv('datasets/loan_test_data.csv')
	# make predictions
	predictions = classification.predict_model(classification_dt, data=test_data_classification)
	# view the predictions
	predictions

view rawprediction_test_pycaret.py hosted with  by GitHub

 

Save and Load the Model

Now, once the model is built and tested, we can save this in the pickle file using the save_model function. Pass the model to be saved and the file name and that’s it:

	# save the model
	classification.save_model(classification_dt, 'decision_tree_1')

view rawpycaret_save_model.py hosted with  by GitHub

We can load this model later on and predict labels on the unseen data:

	# load model
	dt_model = classification.load_model(model_name='decision_tree_1')

view rawpycaret_load_model.py hosted with  by GitHub

 

End Notes

It really is that easy to use. I’ve personally found PyCaret to be quite useful for generating quick results when I’m working with tight timelines.

Practice using it on different types of datasets – you’ll truly grasp it’s utility the more you leverage it! It even supports model deployment on cloud services like AWS and that too with just one line of code.

If you have any suggestions/feedback related to the article, please post them in the comments section below. I look forward to hearing about your experience using PyCaret as well. Happy learning!

Content from : https://www.analyticsvidhya.com/blog/2020/05/pycaret-machine-learning-model-seconds/