What is churn prediction?

Customer Churn rate (also known as attrition rate), in its simplest terms, is a measure of the number of customers ceasing the relationship with their current service provider on the basis of dissatisfaction. Customer churn is when an existing customer, user, player, subscriber or any kind of return client stops doing business or ends the relationship with a company. This could, for example, mean cancellation of a subscription, non-renewal of a contract or service agreement, ending of a membership or Consumer decision to shop at another store/use another service provider.

      Fig 1. Churn Analyses will gauge the satisfaction level of customers (Source)

All companies, irrespective of size or growth, are affected by customers defecting and the following financial consequences. Hence, it is very important to understand what had caused your previously loyal customers to churn away and then translate these learnings into business mechanisms to prevent churn.

ACQUIRING CUSTOMERS IS EXPENSIVE, BUT RETENTION CAN LEAD UP TO 95% INCREASE IN PROFITS

Churn rate is one of the important metrics in evaluating business growth and performance. Businesses spend lots of money in acquiring new customers and studies show that Cost of Customer Acquisition (COCA) has increased over 60% in the last 5 years and continuing to rise at a higher rate. Spending tons of money to acquire new customers and then to have the same customers churn away after 3 months is no good for any business. Therefore, Customer Churn Prevention has become a major part of the Customer Relationship Management (CRM) for a business, as the cost to acquire a new customer is anywhere from 5 to 25 times more expensive than retaining an existing one, according to a Harvard Business Review (Gallo). Studies by Harvard Business School also show that improving customer retention rates by 5% increases profits by 25% to 95%.     

              Fig 2. Customer Churn will have compounded exponential effect on Revenue Loss (Source)

FOCUSING ON HIGH VALUE CUSTOMERS CAN HAVE SIGNIFICANT POSITIVE IMPACT ON PROFITS

Studies shows that in the recent years there has been a steady drop in customer trust among consumers, 81% of customers trust the advice of friends and family over that of a business. Also new customers acquired via such referrals of friends or family spend 200% more and have a 16% higher Lifetime Value than other customers. So, taking good care existing customers can reap more valuable customers in future.

CUSTOMER LOYALTY IS KEY TO GENERATING MORE VALUE WITH TIME

Existing customers are considered as the most valuable asset for a business, and therefore, firms and industries emphasize on keeping the existing customers satisfied, by employing practices. Thus, retaining the existing customers is just as important as attracting and acquiring the new customers.

Also according to Marketing Metrics, the success rate of selling to an existing customer is 60-70%, while the success rate of selling to a new customer is only 5-20%!

The following blog post describes top 25 interesting statistics and facts about customer churn.

Fig 3. Existing customers are more valuable than newly acquired (Source) 

CHURN IS IDENTIFIED BASED ON VARIOUS BENCHMARKS OF CUSTOMER DISENGAGEMENT

Identifying churn and what is causing churn is important to take necessary preventive measure against churn. The way churn is defined can vary across businesses depending upon the type of business and revenue model. Churn could be a customer not re-purchasing your products within a stipulated time interval, could be customer unsubscribing from the service, could be customer uninstalling an app etc., In general terms, churn can be defined as a degree of customer inactivity or disengagement observed over time.

Various indicators convey that a business may be losing customers to churn, like low customer engagement, lower customer usage activity, Increasing customer complaints, Low customer rating & customer satisfaction etc.,

Every business needs to create a set-off benchmarks based on which you can identify the customers who are at risk of churn. Some other ways to identify customer churn are by actively collecting customer feedback and reviews to look for any possible customer dissatisfaction, checking with customer service & support teams to understand any requirements of customers that the company is unable to meet.

HOW CHURN RATE CALCULATION CAN MISLEAD YOUR RETENTION STRATEGY

Basic Churn Rate for a time period calculated using the following formula,

The above calculation though tends to give the initial figures to understand churn, it may not be the right calculation when we may have to keep track of churn rate for certain number of periods.

Consider you are calculating churn for the month of April and May,

Let’s say, you started with 10,000 customers at the start of April and lost 500 of them (5 % of 10,000), gained 5000 new customers in April but lost 125 of those new customers. Customers at the start of May is (10000-500) + (5000-125) = 14375. Now in May let’s say you lost same rate of customers similar to April i.e.., 5% of 14375 = 719, gained 5000 new customers and lost 125 of them.

Now based on above formula Churn Rate in April is 6.25%

Churn rate for month of May comes to be 5.87%

Though the churn is similar in both months as setup above, we see the churn rate dropped in May, this has nothing to do with improving churn performance or retention rate, rather a misreporting of the problem by the above formula.

CHURN CALCULATION THAT IS NORMALIZED FOR COMPANY’S GROWTH & SIZE IS THE RIGHT APPROACH

Apart from that, if the month of April didn’t gain as many customers but only 100, 2 of which churned and May remained same as above, then April would have Churn Rate of 5.02% and May would have Churn Rate of 6.3%. This is because the above calculation is highly influenced by number of new customers that joined during the period. It is important to have a churn calculation that better explains churn behavior that is normalized for growth and size of the company. This calculation will also tend to show wrong churn rates when calculated for different time ranges (Monthly, Quarterly, Annual, etc.,), and no. of. days in the month, like Feb 28 days and May 31 days are also likely to affect the churn rate calculation.

In order to tackle above problems in churn calculation Stephen Noble of Shopify came up with the following churn calculation based on probability,

Using the above formula, we get same 0.05% of Probable Monthly Churn for both months April and May which have similar churn as described by data above. By multiplying Probable Monthly Churn with current customer count we get the estimate count of customers who are likely to churn in next 30 days. This calculation will help you keep track of actual changes, customer behavior and improvement or decline in churn rate.

CUSTOMER CHURN IS A COMMON PROBLEM ACROSS INDUSTRIES

Several studies have been conducted to find the churn rate in various industries. One such study is presented in (Shewan) that shows average customer churn rates in a few common industries in the United States and Europe. Companies lose more than 1.6 Trillion USD per year due to customer churn. Among various sectors studied, the author presented that the highest churn rates are seen in subscriptions of daily newspapers in the US with 60%. Banking and Telcos sector witness around 20-25% and 20-38% churn respectively. Software-as-a-Service (SaaS) companies reported churn rates of 5-7%. According to “2018 Fundraising Effectiveness Survey Report”, another sector that has quite high churn is the not-for-profit organizations with 45.5% in 2018 where Donor retention is the key towards ensuring the future financial stability, as it has been observed that most mid-value, high-value and bequest giving comes from long-term, single-gift donors.

Fig 4. Average Churn Rates Across Various Other Industries (Source)

PREDICTING CHURN USING DATASCIENCE & MACHINE LEARNING

Customer retention is considered as one of the major challenges in modern business. Companies employ a vast range of practices to achieve it, varying from AI based solutions that can predict which customers are more likely to churn, thus retention efforts can be prioritized to focus on the most at risk customers. Churn modelling has been performed using two broad approaches: machine learning techniques and survival analysis. There is no single, fit-for-all methodology that can work in all situations; either machine learning models or survival regression could be appropriate based on the application. Machine learning methods, specifically classification, have widely been popular for predictive analytics due to their high performance and ability to handle complex relationships in data. On the other hand, survival analyses can provide valuable insights into questions such as which customers will churn in a particular time period. In this article, we are majorly focused on machine learning based approaches, however you can find a research paper on survival analyses approach towards the end of the article.

DATA QUALITY IS VERY IMPORTANT & ANY DATA IRREGULARITIES CAN TOPPLE ENTIRE ANALYSIS

In order to perform churn analysis using predictive modelling techniques, the first step is to define a churn metric, whether it is a binary classification problem or there is something more to it? The definition can vary depending on business requirements and problem context. Once the churn metric is defined, the next step is to clean the data, that may involve dealing with missing data, Identifying & dealing with outliers in the, removing the inconsistencies between features e.g., deceased date is populated, but deceased status is false. (Resources to learn more about data cleaning)

EDA & FEATURE ENGINEERING ARE KEY TO A BETTER PERFORMING MACHINE LEARNING MODEL

Next is Exploratory Data Analysis (EDA), where we visually explore the cleaned data to identify all interesting data patterns or correlations of various features with the churn customers. This will give us a brief understanding on what features in the data are better capable of explaining churn and what features are not. With the inputs from EDA we proceed to feature engineering, where we generate additional set of features from the already existing features. Features generated in the Feature engineering stage are supposed to magnify the data patterns leading to churn. This in turn helps the machine learning model to be able to easily learn these data patterns and then predict the customers who are at risk of churn more accurately. (Resources to learn about feature engineering)

For example, Customer behavior with time is a good indicator of weather the customer will churn in future or not. So looking at customer activity during entire customer tenure as a feature may not help model so much, but if we create separate features for customer activity in various time periods like last week, last week to month, last month to last 3 months etc., it will help model identify customers who are on a decreasing trend of activity and then be able to predict risk of churn more accurately.

Figure 2 describes all major steps involved in predictive analytics using machine learning. It starts with collection of data (that may come from multiple sources), followed by pre-processing steps that involve data cleaning. Then Feature generation via feature engineering and feature selection where significant features are retrieved, followed by the actual training process where a machine learning algorithm can be employed. Finally, the model is evaluated through metrics such as Precision, Accuracy, F1-score etc.

For more and detailed understanding about churn prediction, please take a look at the following case study of churn analysis developed by Perceptive Analytics and also a blog post by KDnuggets ‘Customer Churn Prediction using Machine Learning’, detailing about the process of churn prediction.

Fig 5. A Machine Learning based Predictive Analytics

COMMONLY FACED CHALLENGES DURING CHURN ANALYSIS

Data Quality issues can lead your analysis to very strange outcomes. Some of commonly known data quality issues are duplicate data, incomplete data, inconsistent data types, Inconsistent measurement units (Length in feet and width in meters), possibility of human error during the analysis etc.,

Outliers also pay potential threat to any analysis; it is very crucial during the data cleaning stage to identify all the possible outliers and then take necessary action on the same before proceeding with analysis. Outliers in data can skew the complete analysis and its outcomes.

Data Imbalance is another important challenge that needs to be addressed for better model performance. Data imbalance is when the classes in your target variable are unevenly distributed. In churn prediction data, the no of customer who have churned will always be significantly less than customers who have not churned (Ex: 1 in every 10 customers may churn). There are several data over-sampling and under-sampling techniques like Synthetic Minority Oversampling Technique (SMOTE), Adaptive Synthetic Sampling Method (ADASYN) etc., which can be used to tackle the data imbalance challenge. Some of the machine learning techniques like ensemble modelling are inherently capable of handling data imbalance and be able to perform better even without data over-sampling or under-sampling. A by Ghent University to understand more about data imbalance techniques in churn analysis.

Sometimes when you have vast customer data it requires huge computational resources to analyze entire data. That is when you do a sampling of the data to get a sample from the whole data, perform all the analysis & modelling on the sample data to get the insights. Sampling needs to be done very carefully by making sure that the derived sample is a proper representation of population data. There is a chance that sample may miss out patterns related to certain segments of customers who are very prone to churn.

During feature engineering & training the machine learning model one has to make sure that the independent features passed on to the model training are not directly related to dependent feature, For example, dependent column ‘IsChurn’ : 1 for churn and 0 not churn and some independent column ‘CustomerStatus’ : Active and Inactive., there is very good chance that all the inactive customers are churned customers. In this case model will only predict churn only based ‘CustomerStatus’ feature alone but nothing else, which will fail to give us actual understanding of what factors are contributing to churn.

Once the model is trained and deployed to identify customers at risk of churn on real time basis, we need to maintain the trained model with latest data. So, the model won’t miss out on any new emerging customer behavioral trends from the latest data, but learn and to predict more accurately for a long time.

Here we are presenting a few approaches that have been employed to predict customer churn using machine learning and data analytics methodologies:

CHURN IN SAAS INDUSTRY

SAAS IS HOW MODERN BUSINESSES FUNCTION

Software as a Service (SaaS) market mainly consists of consumers purchasing cloud-based software services and solutions on a subscription or pay per use basis. Typically, these SaaS software services can be B2B or B2C, and consumers can access these software services over the internet, mostly via a browser. SaaS offers a wide range of advantages to consumers like lower initial capital requirements, auto updates & maintenance, flexibility, and easy scalability etc., For more about SaaS find this article by Greg Elfrink.

Gartner forecasts global public cloud end-user spending will grow by 47% to $397 billion and Software as a Service (SaaS) is expected to be the largest cloud market segment with $145 billion by the year 2022. Certain services in SaaS are expected to be growing at higher rate than others as shown below,

Image Credit – Hubspot     

CHURN IN SAAS: LESS THAN 4% IS OK, 4% TO 7% IS DECENT & GREATER THAN 7% IS NOT GOOD

On an average SaaS industry enjoys lower customer churn rates when compared to most of other industries, but it does not mean SaaS companies can ignore churn. It is always important to compare business churn rates with industry specific average churn rates. Among SaaS companies, 4%-7% churn rates is considered as decent, anything above that is a potential indicator for you to deep dive. If you are curious to know how your business compares to other similar companies, open benchmarks by baremetrics shows average performance metrics of similar companies based on average revenue per user.

Churn is one of the most important metrics of a SaaS business alongside Monthly Recurring Revenue (MRR) and Annual Recurring Revenue (ARR). No matter how small the churn rate is no SaaS company should ignore churn, when ignored it can consume your recurring revenue to an extent that your SaaS business may no longer be sustainable.

If a SaaS company’s net revenue churn is above 2% each month, it indicates a fault within the company that needs to be fixed. More than two-thirds of SaaS companies had an annual churn rate of 5% or more in a given year.

To get a deeper understanding of churn in SaaS Industry, go through the following blog post which summarizes the results of 6 best Surveys conducted by various individuals on churn in SaaS Industry.

Here’s what is happening to your active subscriptions even with those so-called good churn rates.

Fig. Percentage of Active subscriptions by the end of 12 months (By Hubspot)

REVENUE CHURN IS EQUALLY IF NOT MORE IMPORTANT THAN CUSTOMER CHURN IN SAAS

When it comes to SaaS, there are two types of churn that are important, Customer churn and Revenue Churn. Revenue Churn another type of churn and is the total revenue that business has lost within a given period of time. It can be measured in terms of lost Monthly Recurring Revenue (MRR) or Annual Recurring Revenue (ARR). Revenue churn doesn’t always imply losing customers but only means that revenue has declined. Customer churn only considers the customers who are no longer in association with the business, whereas revenue churn will take into account factors like cancelled customers, downgraded subscriptions and any other factors that are leading to loss of revenue.

For Example, losing a customer with $100/month plan is not same as losing a customer with $1000/month plan, this difference is captured in revenue churn but not in customer churn.

CHURN CALCULATION IN SAAS

There are variety of SaaS companies, and each calculates churn rates differently, all those different approaches are based on and slight variants of simple concept of percentage of customers that leave your service. It is important for a company to make sure that churn rates are calculated properly but at the same time they are not over complicating the calculation.

‘43 ways to calculate SaaS Churn’ by Hubspot will help you understand all the elements that are needed to calculate SaaS churn accurately by keeping it simple at the same time.

GOOD CHURN PLATEAUS WITH TIME, BUT BAD CHURN WILL CONTINUALLY LOSE YOUR CUSTOMERS

Some of the newly acquired customers of yours may leave you not because your product is not up to the mark, but rather just that the product is not very appropriate to their use case. This type of churn is known as good churn. If you have good churn, you may lose customer initially, but eventually you will stop losing customers and the customers that stay are happy with your product.

On the other hand, bad churn is when you keep losing customers continually. The retention continually slopes down to zero and after a certain while everyone who joined at a certain point in time is going to be churn out of your service.

Fig. Good churn Vs Bad Churn

THREE STAGES OF SAAS CHURN AND HOW TO IMPROVE CHURN IN EACH STAGE

SaaS churn can be categorized into three different stages and churn in each of these stages needs to be treated differently.

SHORT-TERM CHURN: TARGETING RIGHT CUSTOMERS IS THE KEY TO REDUCE SHORT TERM CHURN

When customers cancel their subscription within the first one or two months. Churn rates will be higher in this shot term period. This is when the customer will try to figure out core value your product/service has to offer, and they will churn out quickly if they value is not up to their expectation.

Fig. Short Term Churn

Churn in this period is very important and improving short term churn can have great impact on churn rates throughout the customer life cycle. To improve short term churn, it is important to target the right customers and then demonstrate the customers all the core values of product as quickly as possible.

MID-TERM CHURN: KEEP YOUR CUSTOMERS ENGAGED TO REDUCE MID-TERM CHURN

Churn after the first one or two months is called mid-term churn. Customers who have found value of your product in the short-term phase will move on to mid-term phase. In this phase you are looking to slow the churn rate and flatten the churn rate, which tells those customers are continuing to use product.

Fig. Mid Term Churn

In order to improve the mid-term churn, it is important to keep the customers interacting and engaged with your product/service. You can add new set of small features to the product around same core value, refine already existing features, improving customer service to make your product even easier to use.

LONG-TERM CHURN: PRODUCT DEVELOPMENT AND SERVICE IMPROVEMENTS ARE VITAL IN THIS STAGE

Long term churn is when the customer has successfully moved on from mid-term phase and has developed a strong relation with your product/service. Having a good long term churn rates conveys that the customers are really finding value in the product/service that is offered.

Fig. Long Term Churn

One way to improve your long-term churn is by asking your established customers to upgrade to better plans or editions. When there is constant product development with new capabilities and features, upgraded customers will get to experience core value of product once again. You can also reintroduce your new features to some of the inactive accounts to help them know the value of your service better.

By making necessary improvements at every stage of SaaS churn, you are helping customers to understand the true value of your product or service, which helps in generating more Revenue Per User and possibly net negative churn.

Negative churn is when the amount of new revenue from your existing customers upgrades & cross-sells is greater than the revenue that you lose from cancellations and downgrades.

To understand more above negative churn and how to achieve the same, take a look at the following blog post ‘What is Negative Churn’ by baremetrics.

Practical Tips to Improve your SaaS Churn Rates

COMPANIES WITH HIGHER AVERAGE REVENUE PER USER (ARPU) OBSERVE LESS CHURN

We can see that companies with ARPU > $500 have a significantly less revenue churn.

Image Credit – Profitwell

LONGER CONTRACTS REDUCE CHURN

When majority of the customer base is having annual or higher term contracts, such companies observe lower churn rates.

Image Credit – Profitwell

EPERIENCED COMPANIES OBSERVE SIGNIFICANLTY LOWER CHURN THAN OTHERS

Companies which are 3 years old observe  churn rate somewhere in range 4% – 24% where as companies which are 10 years old are observing churn rate between 2% – 4% range.

Image Credit – Profitwell

HAVING TRANSACTIONAL DIFFICULTIES CAN LEAD TO HIGHGER CHURN

Studies show that credit card failure during the payment process has a significant impact on the churn. By fixing payment related issues companies can reduce churn by about 20%-40%.

Image Credit – Profitwell

17 more ways to reduce your SaaS Customer Churn a blog post by Accelerateagency.

CHURN IN TELECOM INDUSTRY

Churn in Telecom industry is when customers cancel or ends the subscription service. Studies show that top 4 telecom carriers in US lose about $260M USD per month. Telecom Industry is one of the major sectors in which a lot of efforts have been employed to determine the factors that can prove vital for customer retention. Study performed on top Canadian telecom providers (BCE & Telus) has identified that cost to retain the old customers is 50 times less than cost to acquire new customer. Several predictive models have been developed to predict the dissatisfied customers who are likely to change the service provider. The major factors of customer dissatisfaction that sometimes result in churn, are poor network quality, call cost, lack of value-added services, support and customer service response time. These factors have been employed with several learners such as support vector machines, neural networks, as decision trees, probabilistic models such as Bayes, etc. Here a brief overview of major works related to churn prediction in the Telecom industry is presented.

A wide variety of machine learning methodologies have been applied to perform customer churn in the Telecom industry. These methodologies can broadly be categorized into Traditional Predictors (such as Decision Tree, Support Vector Machine Neural Networks), Ensemble Based Predictors (Bagging, Boosting, Stacking etc), Predictors with Sampling and Feature Engineering (S&FE), Rule Induction Predictors (RI), Deep Learning based Predictors and Other Miscellaneous Approaches (Misc).

TELECOM CHURN RELATED RESEARCH ARTICLES

One of the earliest work published about Churn Prediction in Telecom was by Mr. Shyam V. Nath, where author presented Naıve Bayesian (NB) on the Teradata Duke Dataset using Customer Demographic Data, reporting an accuracy of 68%. Similarly Xia and Jin [2008] performed experiments on the UCI Dataset and the Home Telecommunication Carrier dataset and achieved an accuracy of 90.9% using SVM Radial Basis Function (RBF) kernel. One of the earlier work in churn prediction using Neural Networks was presented by Sharma and Kumar Panigrahi [2011] who applied a Feed Forward Back Propagation (FFBP) Neural Network on the UCI dataset, achieving an accuracy of 92.35%.

Ensemble techniques have also been employed for churn prediction in Telecom. An ensemble model comprises multiple models that classify the new set of samples by taking vote on the predictions of constituent models. The popular ensemble based models, used for churn prediction in the telecom industry, include Bagging, Boosting and other hybrid models. One of the earliest work reported in the churn prediction in telecom industry using ensemble is Mozer et al. [2000] who explored techniques including Linear Regression, Decision Tree, NN along with Adaptive Boosting (AdaBoost) on a US telecom company dataset. In more recent studies, nested ensemble techniques have also been reported to produce very good results. Mehreen et al, 2018 is one such example, who suggested two such models named as Boosted-Stacked learners and Bagged-Stacked learners, performed experiments publicly available UCI Churn Dataset, reporting the accuracy of  98.4%.

PAPER: The following paper “Customer Churn Prediction in Telecom using Machine Learning in Big Data Platform”, uses Social Network Analysis (SNA) features in the prediction model and demonstrated an increase in performance of the model from 84 to 93.3% when employed on data provided by SyriaTel telecom company. Social Network Analysis (SNA) features are used to summarize the connections between every two customers and build a social network graph(viz) based on the original data.

They used Decision Tree, Random Forest, Gradient Boosted Machine Tree GBM and Extreme Gradient Boosting XGBOOST, of which XGBOOST model with Social Network Analysis (SNA) features has overperformed all other models

Fig. Sample Social Network Graph of Telecom Customers Interactions (each point is a customer)

In the above below, Panel (a) shows the improvement of churn predictive model using Statistical Features related to different historical periods, panel (b) presents the changes in predictive model improvement using SNA Features related to the same historical periods, and panel (c) presents the enhancement of churn predictive model when using both statistical and SNA Features.

                 Fig. Various Model Performance with Different sets of features

Another recent work “Applying over 100 classifiers for churn prediction in telecom companies” is presented by Mr. Debjyoti 2020, who presented an exploratory data analysis report for churn in Telecom, applying a large number of attributes, i.e. 57 attributes of 51,047 customers, with 114 different classifiers, ranging from Decision Tree, SVM to various ensemble based methods. They concluded that ensemble approaches are the best performing approaches in this domain. There are not any notable works that employ deep learning techniques for Customer Churn.

In a recent study, “Telecom churn prediction and used techniques, datasets and performance measures: a review” by Mr.Jain 2020, enlisted the top reason for customer churn in telecom as follows:

• Lack of engagement
• Lack of promotion/or new offers
• Lack of customer service support
• High call rates or SMS charges
• Non-payment bills, fraud, or misuses of services by customers
• Change of location or position

ADDITIONAL RESOURCES: DATASETS AND EXAMPLE TUTORIALS

A few publicly available datasets have been used as a standard in various studies. UCI dataset is the most frequently used dataset. It is artificially created based on the behavior observed in the real world. The UCI dataset has 20 attributes for 5,000 subscribers. The other popular datasets are:

• Churn Prediction Dataset, available at the University of California (UCI) machine learning repository (henceforth mentioned as UCI Dataset),
• Teradata Duke Dataset, a Churn dataset from Teradata Center for Customer Care relationship management at Duke University
• KDD Cup 2009 datasets, the Knowledge Discovery & Data Mining (KDD) Cup 2009 datasets collected from a French telecom company Orange.

There are many exam case studies, tutorials along with datasets that demonstrate how a Exploratory Data Analysis (EDA) and predictive modelling can be performed. A few such Example Tutorials are enlisted here:

• Kaggle: Telco Customer Churn Dataset: The raw data contains 7043 rows (customers) and 21 columns (features). The features mostly comprises Personal and Demographic information such as Gender, SeniorCitizen, Dependants etc and some Billing related information such as monthly charges, total charges, payment method etc.
• Example Tutorial: Classification Analysis on Telco Customer Churn

EDA followed by predictive modelling using Gradient Boosting.

• Example Tutorial: Telecom Churn Prediction.

EDA followed by predictive modelling using SVM, Logistic Regression, Random Forest, AdaBoost, XGBoost

• Kaggle: Telecom Churn Dataset (Cleaned Orange Telecom Customer Churn Dataset): The data contains 667 rows (customers) and 20 columns (features).
• Kaggle: Churn in Telecom’s Dataset: Around 5000 rows (customers) and 21 columns (features). The features mostly comprises Call Records information such as Total Call Minutes, Total Evening Minutes, Total Day Charge etc.
  • Example Tutorial: Customer Churn Analysis
• RStudio : IBM Watson Telecom Churn Analysis :

A detailed churn analysis from scratch on IBM Watson telecom churn data, involving data cleaning, feature engineering, modelling and generating insights with access to all the code.

MOST IMPORTANT FEATURES OF CHURN ANALYSIS IN TELECOM INDUSTRY

The churn datasets in Telecom domain have different attributes (features). In the paper “Customer churn prediction in telecom using machine learning in big data platform” by Ahmed and Afzal 2017, they categorized the features used in research related to Churn prediction into following broad categories

• Customer Demographic Data (CDD)
  • Location Information such as State, Zip, apartment house, official accommodation etc.
  • Personal Information such as Age, number of children, birthday, handset capabilities etc.
• Customer Service Usage Data (CSUD)
  • Call Detail Records such as Call minutes, time and charges, Number of Calls to Customer Service, Number of service calls, Number of competitor service calls,
  • Call Message Records such as Outgoing/incoming number of SMS, Number of Voice-Mail Messages, Message sending times, message, sending times etc.,
• Customer Account Data (CAD)
  • Account Length, International Plan, Voicemail Plan
• Customer Billing and Purchase Data (CBPD)
  • Number of recharges, Total recharged amount, Revenue in months, Total revenue of last year, International calling fees, Count of Overdue Payment etc.,
• Customer Relation Data (CRD).
  • The type, duration of complaints, and the number of days the complaint lasted and if any money was refunded to the customer, estimated time it will take to resolve a complaint by the company and how many days it was delivered late, the type of fault, duration of repairs, number of appointments and number of engineer visits to the site

CHURN IN BANKING SECTOR

Banking sector is among the top 5 industries where we observe higher customer churn. Typically churn rate in banking would vary from 10% to 30%. Competitors would be main reason of customer churn in banking, customers tend to switch banks based on factors like better service, use of latest technology, reliability, Interest rates, service costs, location, customer satisfaction and various other benefits/services provided by bank. (Ex: Discounts on card transactions etc.,). Studies show that identifying the customers who are at risk and reaching out to them at the right time can avoid as much as 11% of customer churn in banking sector.

We can define customer churn in banking sector as one of the following,

• Customer who has closed the bank account.
• Customer who has not closed the account but didn’t make any sort of transactions for a specific time period.
• Customers who only use either 1 or 2 products/services offered by bank out of 10s of products/services.

RELATED RESEARCH PAPERS

Many studies have been performed to analyze churn in the Banking sector through descriptive and predictive analytics.

1. Oyeniyi & A. B. Adeyemo published a paper on Churn Analysis in Banking Sector which describes a early warning mechanism to identify customers who are at risk of churn using K-Means clustering technique & JRip rule based algorithm.

Machine learning based churn prediction models requires lot of manual effort in feature engineering stage, A. B. Adeyemo also published a paper on Customer Churn Prediction using Artificial Neural Networks which eliminates the need of manual feature engineering for churn analysis. The results show an accuracy of 97.53% and ROC of 0.89.

ADDITIONAL RESOURCES: DATASETS AND EXAMPLE TUTORIALS

A few publicly available datasets and tutorials are briefly presented here:

• Kaggle: Predicting Churn for Bank Customers. This data set contains details of a bank’s customers and the target variable is a binary variable reflecting the fact whether the customer left the bank (closed his account) or he continues to be a customer. The raw data contains around 10,000 rows (customers) and 14 columns (features).
  • Example Tutorial: Bank Customer Churn Prediction

EDA followed by predictive modelling using Logistic Regression, SVM, Random Forest and  XGBoost

• Kaggle: Credit Cards Customers Data. This dataset consists of 10,000 rows (customers), 18 features. The data is imbalanced with 16.07% of customers who have churned.
  • Example Tutorial: Bank Churn Data Exploration And Churn Prediction
  • EDA (with up-sampling using SMOTE and PCA) followed by predictive modelling using Random Forest, SVM and AdaBoost. Highest reported accuracy is for Random Forest with 91%

MOST IMPORTANT CHURN RELATED FEATURES IN BANKING INDUSTRY

The most commonly used features in Bank Customer churn prediction are:

• Customer Demographic and Personal Data : Gender, Age, Location, Marital Status
• Customer Survey/Perception Data : Data quantifying how customer appreciates the services or products. Ex : Rating on services, location convenience, Customer opinion on pricing, etc.,
• Customer Account Data (CDD)

Credit Score, Balance, Number of Products, Has Credit Card, Estimated Salary, Total Transaction Amount (12 months), Total Transaction Count (12 months)

• Macro-Economic and Development Indicators : Data related to current economic progress & state of affairs can also play a role on customer behavior.

MISCELLANEOUS CHURN STUDIES

PAPER: CHURN PREDICTION IN MOBILE GAMING APPLICATIONS USING SURVIVAL ENSEMBLES

Survival analytics will focus more on predicting the approximate time frame in which the customer will churn is likely to churn, but the accuracy of the survival analytics is always questionable when compared to classical approaches. Classical regression and classification based churn prediction models will need good amount of churned users data for them to be able to perform better. This paper ‘Churn Prediction in Mobile Social Games’ talks about an approach called Survival Ensembles bringing together the best of both survival and classical methods, to give highly accurate churn prediction with little data.

The paper identifies the probability of customer churning as a function of time (left) and explains the main factors that are causing the churn (right) as show in the following images.

          Fig.  Probability of Churn with Time                                                                 Fig. Feature Importance

EMPLOYEE ATTRITION/CHURN CAN CAUSE A LOSS OF 33% OF THE EMPLOYEE ANNUAL SALARY

Other than customer or client churn, Employee attrition/Churn has also been studied using data analytics techniques. A study by Employee Benefits News on employee retention found that the average cost of losing an employee is a staggering 33% of their annual salary. Organizations spend huge investment for training expenses, both in terms of time and money, in order to make an employee a productive member. Therefore, organization incurs substantial loss when a productive employee quits. Several studies have been reported that make use of advancement in machine learning and other techniques to predict employee attrition and to understand the key variables that influence turnover 

ADDITIONAL RESOURCES: KAGGLE DATASET AND CHURN ANALYSIS TUTORIAL

Employee Attrition (Fictional dataset on HR Employee attrition and performance)

The HR employee attrition data set, comprising 1470 employees (rows) and 35 features (columns).

Make use of the following tutorial on the above employee attrition dataset, which provides a detailed churn analysis from scratch. The author builds & compares three machine learning models based on algorithms like Logistic Regression, Decision tree and Random Forest by tackling the data imbalance using few sampling techniques.

Example Tutorial: Employee Attrition Analysis using R

HOW TO PREVENT CHURN

The key to prevent churn is to Stop customer churn before it even starts.  Once, the probability of customer churn is calculated using data analytics techniques, the key factors that make customers leave can be assessed, followed by (i) targeted loyalty campaigns aimed at the segment with the highest chances of attrition (ii) focus on valuable customers and let go jumpers (iii) use more flexible marketing and (iv) sales campaigns and change products to cover customer needs.

Most common pitfall for businesses around the world is that they think if they have great product or service to offer customer retention will happen naturally and they do not have to put in any additional effort to retain. While it maybe true to some extent, it is only works in shorter period, and after a while customers will start leaving you.(read more here)

Fig. Why do customers leave a company (Source)

A study summarizes the method to improve churn prevention into following list of key points

1. Analyze why churn occurs
2. Engage with your customers (relation marketing through emails, social listening, feedback surveys)
3. Educate the customer (free trainings, webinars, video tutorials, and product demos)
4. Personalize customer experience
5. Know who is at risk (Find out which customers have not been contacted for a while)
6. Offer incentives discounts and special offers
7. Target the right audience (better to target those who appreciate the long-term value of products)
8. Give outstanding customer service and support
9. Pay attention to complaints (Complaints are like tips of the icebergs – they suggest that the bigger part of the problem is hidden from the view).
10. Make your best people deal with cancellations
11. Flaunt your competitive advantages (How different are you from your competitors? What makes you stand out? What will your customers lose if they decide to quit?
12. Offer long term contracts (extending your customers’ commitment from monthly to yearly etc.,)

PAPER: MANGING CHURN TO MAXIMIZE PROFITS

Customer retention strategies typically involve two steps, 1) Ranking customers based on their propensity to churn and 2) then offering retention incentives to a subset of customers who are at the top of ranking. The implicit assumption is that this process would maximize firm’s profits by targeting customers who are most likely to churn. However, current marketing research and practice aims at maximizing the correct classification of churners and non-churners. Profit from targeting a customer depends on not only a customer’s propensity to churn, but also on his/her spend or value, his/her probability of responding to retention offers, as well as the cost of these offers. Overall profit of the firm also depends on the number of customers the firm decides to target for its retention campaign. proposes a binary classification based predictive model that identifies the right customers to target considering all the above-mentioned factors and then optimally create retention strategy which leads to, on average, a 115% improvement in profit compared to other methods.

Fig. Study1: Television Campaign                          Fig. Study 2: Special Interest Organization Campaign

The above figure describes how in both the studies the approach proposed in the paper (Profit Loss) is achieving better profitability compared to other approaches. Also in Study 1, we can notice the profitability of suggested approach is maximum at target size of 60%. The straight dashed line is the average profitability when our target size is 100%.

A successful customer retention strategy is very important to the overall success of any business. It can save valuable resources, can reap better profits, gain customer loyalty, word of mouth marketing, higher referrals, larger repeat business transactions and much more.

Conclusion

With immense digitization of businesses across the world consumers have access to several service providers. In such a competitive environment it takes a lot of effort & resources to acquire new customers. Also cost to the customer for switching service provider are gradually decreasing which leads to higher churn rates. An effective churn analysis mechanism to identify the customers who are at risk of churn, understanding the reasons that are causing customer churn and adopting  a proactive customer retention strategies to prevent churn is very crucial to the performance of any business.

We can predict churn by analyzing behavioral and transactional data of customers. Mining their data, to find patterns that are specific to customers who have already churned, can help you identify those customers, who are likely to churn. In order for a firm to increase the retention rate and reduce the churn rate, the firm has to focus on the customer satisfaction and making user interactions (transaction attributes), more user friendly and hassle free for its customers.

Churn prediction can be used within your business. It is one of the key components, which in conjunction with lifetime value of customers (CLV) can help you take marketing decisions that is customized at a customer level. One can compare churn rates in different customer segments (high CLV segment and low CLV segment), and in turn we can take customized decisions to retain highly profitable customers and also not to spend more than required resources on retaining customers who are least profitable to the business.

Bibliography

Ahmad, Abdelrahim Kasem. 2019. “Customer churn prediction in telecom using machine learning in big data platform.” Journal of Big Data 6 (1).

Ahmed, Mehreen, and Hammad Afzal. 2017. “A survey of evolution in predictive models and impacting factors in customer churn.” {Advances in Data Science and Adaptive Analysis 9 (3).

Debjyoti, Adhikary. 2020. “Applying over 100 classifiers for churn prediction in telecom companies.” Multimedia Tools and Applications.

Gallo, Amy. n.d. “The Value of Keeping the Right Customers.” Harvard Business Review. Accessed 3 28, 2014. https://hbr.org/2014/10/the-value-of-keeping-the-right-customers.

Hadden, John. 2006. “Churn prediction: Does technology matter.” International Journal of Intelligent Technology 1 (2).

Jain, Hemlata. 2020. “Telecom churn prediction and used techniques, datasets and performance measures: a review.” Telecommunication Systems.

Levis, Bill. n.d. “2018 Fundraising Effectiveness Survey Report.” Accessed 3 28, 2021. https://afpfep.org/wp-content/uploads/2018/04/2018-Fundraising-Effectiveness-Survey-Report.pdf.

Nath, Shyam V. 2003. “Customer churn analysis in the wireless industry: A data mining approach.” Proceedings-annual meeting of the decision sciences institute} 561.

Shewan, D. n.d. “How to Calculate (And Lower!) Your Customer Churn Rate.” Accessed 3 28, 2021. https://www.wordstream.com/blog/ws/2014/05/12/customer-churn.

Steven H. Noble, Shopify Engineering 2011. https://shopify.engineering/defining-churn-rate-no-really-this-actually-requires-an-entire-blog-post

Comments :

• CleverTap Article on APP Churn (Add It) –

References

1. https://hbr.org/2014/10/the-value-of-keeping-the-right-customers
2. http://www2.bain.com/Images/BB_Prescription_cutting_costs.pdf
3. https://www.netigate.net/articles/customer-satisfaction/customer-churn-meaning/