The purpose of this project is to investigate the churn of Beta Bank customers. The bank has noticed customers are slowly leaving, month over month. The most cost effective solution is to retain existing customers, rather than recruit new customers. The bank needs a model that will predict whether a customer will leave the bank soon. The model will be trained on data from clients' past behavior and termination of contracts with the bank. The goal of the project is to build a model with an F1 score of at least 0.59.
This is a binary classification problem, therefore, we will be focussing on that class of modeling. This is the case because the target is categorical: churn or not. The F1 score is an appropriate indicator of our model, because we care about both precision and recall. We care about precision because we want to capture as many true positive occurrences of churning. Recall is also crucial, as we want to capture as many occurrences of churning as possible.
We were able to make a model that could predict whether a Beta Bank customer would churn. Using the key numeric features, and a few categorical features, we were able to train our model to achieve an F1 score of 0.594 on our test set. This F1 score is above the metric of a random model, which would have an F1 score of 0.5. Looking at other metrics, we see our model is relatively accurate, at 0.851 with the test data. Based on the results of the classification report, we had greater precision and recall predicting when a customer would not churn. The model was fairly precise in predicting churn, and weak when it came to recalling churn. This means the model was not good at predicting most of the customers that would churn. Since F1 is a measure of precision and accuracy, our F1 score appears low. However, we achieved our 0.59 target with our test set. Furthermore, our AUC-ROC metric determines how much our model differs from a random model with an AUC-ROC of 0.50. An AUC-ROC score of 0.848 means our model is better at predicting churn than chance. Overall, Beta Bank can use this model to predict which customers will not churn. From there, they can determine the features that may contribute to a customer staying with them, in order to retain much more customers. Alternatively, this model can predict a decent portion of customers that would churn. We suggest more data is collected, with more features that could contribute to customers churning. Another way of improving the model would be to be to see if the max depth approaches a limit. Data without missing tenures would also aid in improving the results, as well as more balanced data on churned customers.