Let’s start by reading in the dataset so we can investigate all customers’ attributes.
churn <- read.csv('~/Desktop/Telco Customer Churn.csv')
head(churn)
## customerID gender SeniorCitizen Partner Dependents tenure PhoneService
## 1 7590-VHVEG Female 0 Yes No 1 No
## 2 5575-GNVDE Male 0 No No 34 Yes
## 3 3668-QPYBK Male 0 No No 2 Yes
## 4 7795-CFOCW Male 0 No No 45 No
## 5 9237-HQITU Female 0 No No 2 Yes
## 6 9305-CDSKC Female 0 No No 8 Yes
## MultipleLines InternetService OnlineSecurity OnlineBackup
## 1 No phone service DSL No Yes
## 2 No DSL Yes No
## 3 No DSL Yes Yes
## 4 No phone service DSL Yes No
## 5 No Fiber optic No No
## 6 Yes Fiber optic No No
## DeviceProtection TechSupport StreamingTV StreamingMovies Contract
## 1 No No No No Month-to-month
## 2 Yes No No No One year
## 3 No No No No Month-to-month
## 4 Yes Yes No No One year
## 5 No No No No Month-to-month
## 6 Yes No Yes Yes Month-to-month
## PaperlessBilling PaymentMethod MonthlyCharges TotalCharges
## 1 Yes Electronic check 29.85 29.85
## 2 No Mailed check 56.95 1889.50
## 3 Yes Mailed check 53.85 108.15
## 4 No Bank transfer (automatic) 42.30 1840.75
## 5 Yes Electronic check 70.70 151.65
## 6 Yes Electronic check 99.65 820.50
## Churn
## 1 No
## 2 No
## 3 Yes
## 4 No
## 5 Yes
## 6 Yes
dim(churn)
## [1] 7043 21
We have access to information on 7043 clients, with 21 attributes.
The dataset provides us the features below:
sapply(churn, function(x) sum(is.na(x)))
## customerID gender SeniorCitizen Partner
## 0 0 0 0
## Dependents tenure PhoneService MultipleLines
## 0 0 0 0
## InternetService OnlineSecurity OnlineBackup DeviceProtection
## 0 0 0 0
## TechSupport StreamingTV StreamingMovies Contract
## 0 0 0 0
## PaperlessBilling PaymentMethod MonthlyCharges TotalCharges
## 0 0 0 11
## Churn
## 0
The dataset has 11 missing values, all of them under TotalCharges. An error might have occurred in the collection process, which prevented us from getting them correctly. But we have MonthlyCharges, which might be enough for the purpose of this exercise. In fact, if these two variables are highly correlated we can only use one of them to avoid multicollinearity.
library(corrplot)
complete <- churn[complete.cases(churn), ]
charges <- complete[,19:20]
corr.matrix <- cor(charges)
corrplot(corr.matrix, main="\nCorrelation Plot for Total & Monthly Charges", method="number")
As Monthly and Total Charges are correlated and we lack data for Total Charges, instead of getting rid of the rows with missing values, I will leave Total Charges out of the equation.
Regarding the tenure variable, it ranges from 0 to 72months. So, we can group them into five tenure groups:
group_tenure <- function(tenure){
if (tenure >= 0 & tenure <= 12){
return('0-12 Month')
}else if(tenure > 12 & tenure <= 24){
return('12-24 Month')
}else if (tenure > 24 & tenure <= 48){
return('24-48 Month')
}else if (tenure > 48 & tenure <=60){
return('48-60 Month')
}else if (tenure > 60){
return('> 60 Month')
}
}
churn$tenure_group <- sapply(churn$tenure,group_tenure)
churn$tenure_group <- as.factor(churn$tenure_group)
SeniorCitizen is a binary feature. Let’s change it in order to follow the same format as the remaining variables.
churn$SeniorCitizen <- as.factor(ifelse(churn$SeniorCitizen==1,"Yes","No"))
Interestingly, there are a few attributes which should also have a “Yes/No” output but have different ways of providing the negative answer. Therefore, we need to change “No internet service”" to “No”" for six columns: “OnlineSecurity”“,”OnlineBackup“,”DeviceProtection“,”TechSupport“,”StreamingTV“,”StreamingMovies“.
library(plyr)
for(i in 1:ncol(churn[,10:15])) {
churn[,10:15][,i] <- as.factor(mapvalues
(churn[,10:15][,i], from =c("No internet service"),to=c("No")))
}
This situation happens for for MultipleLines too, so let’s take the same approach.
churn$MultipleLines <- as.factor(mapvalues(churn$MultipleLines,
from=c("No phone service"),
to=c("No")))
Before moving on to some exploratory analysis, we can ditch unnecessary variables which will add nothing to our predicting models.
library(dplyr)
churn <- churn %>% select(-customerID, -tenure, -TotalCharges)
Below we can see how our categorical variables behave depending on churn outcomes.
From the graphs above, we can easily see that, for example, gender doesn’t add much value to our prediction. Conversely, Month-to-Month contracts tend to churn more often than one or two year. Also, paperless billing customers are more likely to cancel their subscriptions. How do MonthlyCharges relate to churn?
It seems we face a few churn risks for clients who have higher MonthlyCharges.
But let’s further analyse churn prediction using Logistic Regression.
library(caret)
intrain<- createDataPartition(churn$Churn,p=0.7,list=FALSE)
set.seed(2019)
training<- churn[intrain,]
testing<- churn[-intrain,]
LogModel <- glm(Churn ~ .,family=binomial(link="logit"),data=training)
print(summary(LogModel))
##
## Call:
## glm(formula = Churn ~ ., family = binomial(link = "logit"), data = training)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -1.9857 -0.6827 -0.2928 0.6859 3.1216
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -0.99659 0.98632 -1.010 0.312298
## genderMale -0.04015 0.07749 -0.518 0.604365
## SeniorCitizenYes 0.26708 0.09998 2.671 0.007558
## PartnerYes -0.03314 0.09177 -0.361 0.717996
## DependentsYes -0.08449 0.10633 -0.795 0.426836
## PhoneServiceYes 0.19698 0.78252 0.252 0.801250
## MultipleLinesYes 0.47184 0.21222 2.223 0.026190
## InternetServiceFiber optic 1.90776 0.96143 1.984 0.047222
## InternetServiceNo -1.74627 0.96994 -1.800 0.071799
## OnlineSecurityYes -0.13231 0.21406 -0.618 0.536516
## OnlineBackupYes 0.05669 0.21235 0.267 0.789508
## DeviceProtectionYes 0.16674 0.21220 0.786 0.432000
## TechSupportYes -0.06662 0.21750 -0.306 0.759377
## StreamingTVYes 0.61802 0.39388 1.569 0.116631
## StreamingMoviesYes 0.70870 0.39431 1.797 0.072283
## ContractOne year -0.76509 0.12783 -5.985 2.16e-09
## ContractTwo year -1.72042 0.21964 -7.833 4.77e-15
## PaperlessBillingYes 0.31241 0.08940 3.495 0.000475
## PaymentMethodCredit card (automatic) -0.03450 0.13535 -0.255 0.798793
## PaymentMethodElectronic check 0.27402 0.11298 2.425 0.015293
## PaymentMethodMailed check -0.04294 0.13549 -0.317 0.751291
## MonthlyCharges -0.03919 0.03821 -1.026 0.305110
## tenure_group0-12 Month 1.81837 0.20358 8.932 < 2e-16
## tenure_group12-24 Month 1.01517 0.19952 5.088 3.62e-07
## tenure_group24-48 Month 0.53022 0.18381 2.885 0.003918
## tenure_group48-60 Month 0.30053 0.19940 1.507 0.131764
##
## (Intercept)
## genderMale
## SeniorCitizenYes **
## PartnerYes
## DependentsYes
## PhoneServiceYes
## MultipleLinesYes *
## InternetServiceFiber optic *
## InternetServiceNo .
## OnlineSecurityYes
## OnlineBackupYes
## DeviceProtectionYes
## TechSupportYes
## StreamingTVYes
## StreamingMoviesYes .
## ContractOne year ***
## ContractTwo year ***
## PaperlessBillingYes ***
## PaymentMethodCredit card (automatic)
## PaymentMethodElectronic check *
## PaymentMethodMailed check
## MonthlyCharges
## tenure_group0-12 Month ***
## tenure_group12-24 Month ***
## tenure_group24-48 Month **
## tenure_group48-60 Month
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 5707.1 on 4930 degrees of freedom
## Residual deviance: 4107.7 on 4905 degrees of freedom
## AIC: 4159.7
##
## Number of Fisher Scoring iterations: 6
According to the Logistic Regression summary above, the top three most-relevant features include Contract, tenure_group and PaperlessBilling.
anova(LogModel, test="Chisq")
## Analysis of Deviance Table
##
## Model: binomial, link: logit
##
## Response: Churn
##
## Terms added sequentially (first to last)
##
##
## Df Deviance Resid. Df Resid. Dev Pr(>Chi)
## NULL 4930 5707.1
## gender 1 0.25 4929 5706.8 0.6140666
## SeniorCitizen 1 113.28 4928 5593.5 < 2.2e-16 ***
## Partner 1 111.40 4927 5482.1 < 2.2e-16 ***
## Dependents 1 30.76 4926 5451.4 2.921e-08 ***
## PhoneService 1 0.08 4925 5451.3 0.7775022
## MultipleLines 1 10.10 4924 5441.2 0.0014788 **
## InternetService 2 477.62 4922 4963.6 < 2.2e-16 ***
## OnlineSecurity 1 160.22 4921 4803.4 < 2.2e-16 ***
## OnlineBackup 1 68.06 4920 4735.3 < 2.2e-16 ***
## DeviceProtection 1 42.20 4919 4693.1 8.245e-11 ***
## TechSupport 1 70.42 4918 4622.7 < 2.2e-16 ***
## StreamingTV 1 0.35 4917 4622.3 0.5552851
## StreamingMovies 1 0.99 4916 4621.4 0.3207921
## Contract 2 312.81 4914 4308.5 < 2.2e-16 ***
## PaperlessBilling 1 12.70 4913 4295.8 0.0003647 ***
## PaymentMethod 3 26.94 4910 4268.9 6.057e-06 ***
## MonthlyCharges 1 0.93 4909 4268.0 0.3348119
## tenure_group 4 160.29 4905 4107.7 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Analyzing the deviance table we can see the drop in deviance when adding each variable one at a time. Adding InternetService, Contract, OnlineSecurity and tenure_group significantly reduces the residual deviance. Assessing the predictive ability of our model…
testing$Churn <- as.character(testing$Churn)
testing$Churn[testing$Churn=="No"] <- "0"
testing$Churn[testing$Churn=="Yes"] <- "1"
fitted.results <- predict(LogModel,newdata=testing,type='response')
fitted.results <- ifelse(fitted.results > 0.5,1,0)
misClasificError <- mean(fitted.results != testing$Churn)
print(paste('Logistic Regression Accuracy',1-misClasificError))
## [1] "Logistic Regression Accuracy 0.803030303030303"
Although accuracy is good, I believe we can do as good as this but playing with less predictors. I will use the stepwise method to reduce the number of variables at stake.
LogModel2 <- step(LogModel)
## Start: AIC=4159.67
## Churn ~ gender + SeniorCitizen + Partner + Dependents + PhoneService +
## MultipleLines + InternetService + OnlineSecurity + OnlineBackup +
## DeviceProtection + TechSupport + StreamingTV + StreamingMovies +
## Contract + PaperlessBilling + PaymentMethod + MonthlyCharges +
## tenure_group
##
## Df Deviance AIC
## - PhoneService 1 4107.7 4157.7
## - OnlineBackup 1 4107.7 4157.7
## - TechSupport 1 4107.8 4157.8
## - Partner 1 4107.8 4157.8
## - gender 1 4107.9 4157.9
## - OnlineSecurity 1 4108.1 4158.1
## - DeviceProtection 1 4108.3 4158.3
## - Dependents 1 4108.3 4158.3
## - MonthlyCharges 1 4108.7 4158.7
## <none> 4107.7 4159.7
## - InternetService 2 4111.9 4159.9
## - StreamingTV 1 4110.1 4160.1
## - StreamingMovies 1 4110.9 4160.9
## - MultipleLines 1 4112.6 4162.6
## - SeniorCitizen 1 4114.8 4164.8
## - PaymentMethod 3 4120.5 4166.5
## - PaperlessBilling 1 4119.9 4169.9
## - Contract 2 4194.4 4242.4
## - tenure_group 4 4268.0 4312.0
##
## Step: AIC=4157.73
## Churn ~ gender + SeniorCitizen + Partner + Dependents + MultipleLines +
## InternetService + OnlineSecurity + OnlineBackup + DeviceProtection +
## TechSupport + StreamingTV + StreamingMovies + Contract +
## PaperlessBilling + PaymentMethod + MonthlyCharges + tenure_group
##
## Df Deviance AIC
## - OnlineBackup 1 4107.7 4155.7
## - Partner 1 4107.9 4155.9
## - gender 1 4108.0 4156.0
## - Dependents 1 4108.4 4156.4
## - TechSupport 1 4108.8 4156.8
## - DeviceProtection 1 4109.1 4157.1
## <none> 4107.7 4157.7
## - OnlineSecurity 1 4110.5 4158.5
## - SeniorCitizen 1 4114.8 4162.8
## - PaymentMethod 3 4120.6 4164.6
## - PaperlessBilling 1 4120.0 4168.0
## - MultipleLines 1 4123.6 4171.6
## - MonthlyCharges 1 4123.7 4171.7
## - StreamingTV 1 4127.3 4175.3
## - StreamingMovies 1 4135.2 4183.2
## - InternetService 2 4169.2 4215.2
## - Contract 2 4194.5 4240.5
## - tenure_group 4 4268.1 4310.1
##
## Step: AIC=4155.74
## Churn ~ gender + SeniorCitizen + Partner + Dependents + MultipleLines +
## InternetService + OnlineSecurity + DeviceProtection + TechSupport +
## StreamingTV + StreamingMovies + Contract + PaperlessBilling +
## PaymentMethod + MonthlyCharges + tenure_group
##
## Df Deviance AIC
## - Partner 1 4107.9 4153.9
## - gender 1 4108.0 4154.0
## - Dependents 1 4108.4 4154.4
## - TechSupport 1 4108.9 4154.9
## - DeviceProtection 1 4109.1 4155.1
## <none> 4107.7 4155.7
## - OnlineSecurity 1 4110.6 4156.6
## - SeniorCitizen 1 4114.8 4160.8
## - PaymentMethod 3 4120.6 4162.6
## - PaperlessBilling 1 4120.1 4166.1
## - MultipleLines 1 4124.0 4170.0
## - MonthlyCharges 1 4125.9 4171.9
## - StreamingTV 1 4128.1 4174.1
## - StreamingMovies 1 4136.3 4182.3
## - InternetService 2 4174.0 4218.0
## - Contract 2 4194.5 4238.5
## - tenure_group 4 4275.2 4315.2
##
## Step: AIC=4153.87
## Churn ~ gender + SeniorCitizen + Dependents + MultipleLines +
## InternetService + OnlineSecurity + DeviceProtection + TechSupport +
## StreamingTV + StreamingMovies + Contract + PaperlessBilling +
## PaymentMethod + MonthlyCharges + tenure_group
##
## Df Deviance AIC
## - gender 1 4108.2 4152.2
## - Dependents 1 4108.9 4152.9
## - TechSupport 1 4109.0 4153.0
## - DeviceProtection 1 4109.2 4153.2
## <none> 4107.9 4153.9
## - OnlineSecurity 1 4110.7 4154.7
## - SeniorCitizen 1 4114.8 4158.8
## - PaymentMethod 3 4120.7 4160.7
## - PaperlessBilling 1 4120.2 4164.2
## - MultipleLines 1 4124.1 4168.1
## - MonthlyCharges 1 4126.0 4170.0
## - StreamingTV 1 4128.2 4172.2
## - StreamingMovies 1 4136.4 4180.4
## - InternetService 2 4174.1 4216.1
## - Contract 2 4194.6 4236.6
## - tenure_group 4 4282.6 4320.6
##
## Step: AIC=4152.15
## Churn ~ SeniorCitizen + Dependents + MultipleLines + InternetService +
## OnlineSecurity + DeviceProtection + TechSupport + StreamingTV +
## StreamingMovies + Contract + PaperlessBilling + PaymentMethod +
## MonthlyCharges + tenure_group
##
## Df Deviance AIC
## - Dependents 1 4109.2 4151.2
## - TechSupport 1 4109.3 4151.3
## - DeviceProtection 1 4109.5 4151.5
## <none> 4108.2 4152.2
## - OnlineSecurity 1 4110.9 4152.9
## - SeniorCitizen 1 4115.1 4157.1
## - PaymentMethod 3 4121.0 4159.0
## - PaperlessBilling 1 4120.5 4162.5
## - MultipleLines 1 4124.3 4166.3
## - MonthlyCharges 1 4126.3 4168.3
## - StreamingTV 1 4128.5 4170.5
## - StreamingMovies 1 4136.7 4178.7
## - InternetService 2 4174.4 4214.4
## - Contract 2 4194.8 4234.8
## - tenure_group 4 4282.9 4318.9
##
## Step: AIC=4151.24
## Churn ~ SeniorCitizen + MultipleLines + InternetService + OnlineSecurity +
## DeviceProtection + TechSupport + StreamingTV + StreamingMovies +
## Contract + PaperlessBilling + PaymentMethod + MonthlyCharges +
## tenure_group
##
## Df Deviance AIC
## - TechSupport 1 4110.4 4150.4
## - DeviceProtection 1 4110.6 4150.6
## <none> 4109.2 4151.2
## - OnlineSecurity 1 4112.1 4152.1
## - SeniorCitizen 1 4117.5 4157.5
## - PaymentMethod 3 4122.2 4158.2
## - PaperlessBilling 1 4121.8 4161.8
## - MultipleLines 1 4125.5 4165.5
## - MonthlyCharges 1 4127.7 4167.7
## - StreamingTV 1 4129.7 4169.7
## - StreamingMovies 1 4138.2 4178.2
## - InternetService 2 4176.5 4214.5
## - Contract 2 4197.8 4235.8
## - tenure_group 4 4288.0 4322.0
##
## Step: AIC=4150.36
## Churn ~ SeniorCitizen + MultipleLines + InternetService + OnlineSecurity +
## DeviceProtection + StreamingTV + StreamingMovies + Contract +
## PaperlessBilling + PaymentMethod + MonthlyCharges + tenure_group
##
## Df Deviance AIC
## - DeviceProtection 1 4111.9 4149.9
## <none> 4110.4 4150.4
## - OnlineSecurity 1 4112.9 4150.9
## - SeniorCitizen 1 4118.9 4156.9
## - PaymentMethod 3 4123.9 4157.9
## - PaperlessBilling 1 4122.8 4160.8
## - MultipleLines 1 4129.0 4167.0
## - StreamingTV 1 4133.0 4171.0
## - MonthlyCharges 1 4135.9 4173.9
## - StreamingMovies 1 4142.5 4180.5
## - InternetService 2 4190.5 4226.5
## - Contract 2 4204.5 4240.5
## - tenure_group 4 4291.8 4323.8
##
## Step: AIC=4149.95
## Churn ~ SeniorCitizen + MultipleLines + InternetService + OnlineSecurity +
## StreamingTV + StreamingMovies + Contract + PaperlessBilling +
## PaymentMethod + MonthlyCharges + tenure_group
##
## Df Deviance AIC
## <none> 4111.9 4149.9
## - OnlineSecurity 1 4115.1 4151.1
## - SeniorCitizen 1 4120.7 4156.7
## - PaymentMethod 3 4125.6 4157.6
## - PaperlessBilling 1 4124.3 4160.3
## - MultipleLines 1 4129.3 4165.3
## - StreamingTV 1 4133.5 4169.5
## - MonthlyCharges 1 4136.3 4172.3
## - StreamingMovies 1 4143.1 4179.1
## - InternetService 2 4192.2 4226.2
## - Contract 2 4204.9 4238.9
## - tenure_group 4 4292.2 4322.2
print(summary(LogModel2))
##
## Call:
## glm(formula = Churn ~ SeniorCitizen + MultipleLines + InternetService +
## OnlineSecurity + StreamingTV + StreamingMovies + Contract +
## PaperlessBilling + PaymentMethod + MonthlyCharges + tenure_group,
## family = binomial(link = "logit"), data = training)
##
## Deviance Residuals:
## Min 1Q Median 3Q Max
## -2.0058 -0.6806 -0.2923 0.6866 3.1200
##
## Coefficients:
## Estimate Std. Error z value Pr(>|z|)
## (Intercept) -1.292519 0.349402 -3.699 0.000216
## SeniorCitizenYes 0.290105 0.097780 2.967 0.003008
## MultipleLinesYes 0.422040 0.101836 4.144 3.41e-05
## InternetServiceFiber optic 1.687526 0.211182 7.991 1.34e-15
## InternetServiceNo -1.498856 0.201055 -7.455 8.99e-14
## OnlineSecurityYes -0.186529 0.106079 -1.758 0.078680
## StreamingTVYes 0.525684 0.113596 4.628 3.70e-06
## StreamingMoviesYes 0.624623 0.112759 5.539 3.03e-08
## ContractOne year -0.774517 0.127155 -6.091 1.12e-09
## ContractTwo year -1.749369 0.216986 -8.062 7.50e-16
## PaperlessBillingYes 0.313296 0.089238 3.511 0.000447
## PaymentMethodCredit card (automatic) -0.033058 0.135158 -0.245 0.806773
## PaymentMethodElectronic check 0.281677 0.112766 2.498 0.012493
## PaymentMethodMailed check -0.044116 0.135223 -0.326 0.744239
## MonthlyCharges -0.029536 0.005988 -4.933 8.11e-07
## tenure_group0-12 Month 1.817088 0.195198 9.309 < 2e-16
## tenure_group12-24 Month 1.007002 0.194457 5.179 2.24e-07
## tenure_group24-48 Month 0.525052 0.181220 2.897 0.003764
## tenure_group48-60 Month 0.296959 0.198692 1.495 0.135025
##
## (Intercept) ***
## SeniorCitizenYes **
## MultipleLinesYes ***
## InternetServiceFiber optic ***
## InternetServiceNo ***
## OnlineSecurityYes .
## StreamingTVYes ***
## StreamingMoviesYes ***
## ContractOne year ***
## ContractTwo year ***
## PaperlessBillingYes ***
## PaymentMethodCredit card (automatic)
## PaymentMethodElectronic check *
## PaymentMethodMailed check
## MonthlyCharges ***
## tenure_group0-12 Month ***
## tenure_group12-24 Month ***
## tenure_group24-48 Month **
## tenure_group48-60 Month
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## (Dispersion parameter for binomial family taken to be 1)
##
## Null deviance: 5707.1 on 4930 degrees of freedom
## Residual deviance: 4111.9 on 4912 degrees of freedom
## AIC: 4149.9
##
## Number of Fisher Scoring iterations: 6
anova(LogModel2, test="Chisq")
## Analysis of Deviance Table
##
## Model: binomial, link: logit
##
## Response: Churn
##
## Terms added sequentially (first to last)
##
##
## Df Deviance Resid. Df Resid. Dev Pr(>Chi)
## NULL 4930 5707.1
## SeniorCitizen 1 113.37 4929 5593.7 < 2.2e-16 ***
## MultipleLines 1 3.66 4928 5590.0 0.0556351 .
## InternetService 2 502.28 4926 5087.8 < 2.2e-16 ***
## OnlineSecurity 1 200.67 4925 4887.1 < 2.2e-16 ***
## StreamingTV 1 12.56 4924 4874.5 0.0003940 ***
## StreamingMovies 1 2.71 4923 4871.8 0.0994258 .
## Contract 2 497.22 4921 4374.6 < 2.2e-16 ***
## PaperlessBilling 1 13.25 4920 4361.4 0.0002731 ***
## PaymentMethod 3 34.32 4917 4327.0 1.693e-07 ***
## MonthlyCharges 1 34.88 4916 4292.2 3.515e-09 ***
## tenure_group 4 180.21 4912 4111.9 < 2.2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
fitted.results2 <- predict(LogModel2,newdata=testing,type='response')
fitted.results2 <- ifelse(fitted.results2 > 0.5,1,0)
misClasificError2 <- mean(fitted.results2 != testing$Churn)
print(paste('Logistic Regression Accuracy',1-misClasificError2))
## [1] "Logistic Regression Accuracy 0.803503787878788"
We managed to keep accuracy high at , using less predictors. Now let’s see how Decision Trees would predict using only these three features.
library(party)
tree <- ctree(Churn~Contract+tenure_group+InternetService, training)
plot(tree)
pred_tree <- predict(tree, testing)
print("Confusion Matrix for Decision Tree"); table(Predicted = pred_tree, Actual = testing$Churn)
## [1] "Confusion Matrix for Decision Tree"
## Actual
## Predicted 0 1
## No 1410 315
## Yes 142 245
p1 <- predict(tree, training)
tab1 <- table(Predicted = p1, Actual = training$Churn)
tab2 <- table(Predicted = pred_tree, Actual = testing$Churn)
print(paste('Decision Tree Accuracy',sum(diag(tab2))/sum(tab2)))
## [1] "Decision Tree Accuracy 0.783617424242424"
Accuracy slightly dropped when compared to Logistic Regression. But we were using only three predictors. Let’s see how better we can do by using Random Forest.
library(randomForest)
rfModel <- randomForest(Churn ~., data = training)
print(rfModel)
##
## Call:
## randomForest(formula = Churn ~ ., data = training)
## Type of random forest: classification
## Number of trees: 500
## No. of variables tried at each split: 4
##
## OOB estimate of error rate: 21.13%
## Confusion matrix:
## No Yes class.error
## No 3239 383 0.1057427
## Yes 659 650 0.5034377
The error rate seems to be higher when predicting “Yes”. We can also see it in the confusion matrix below:
pred_rf <- predict(rfModel, testing)
testing$Churn2 <- as.factor(ifelse(testing$Churn=="1","Yes","No"))
caret::confusionMatrix(pred_rf, testing$Churn2)
## Confusion Matrix and Statistics
##
## Reference
## Prediction No Yes
## No 1396 278
## Yes 156 282
##
## Accuracy : 0.7945
## 95% CI : (0.7766, 0.8116)
## No Information Rate : 0.7348
## P-Value [Acc > NIR] : 1.072e-10
##
## Kappa : 0.4332
## Mcnemar's Test P-Value : 6.315e-09
##
## Sensitivity : 0.8995
## Specificity : 0.5036
## Pos Pred Value : 0.8339
## Neg Pred Value : 0.6438
## Prevalence : 0.7348
## Detection Rate : 0.6610
## Detection Prevalence : 0.7926
## Balanced Accuracy : 0.7015
##
## 'Positive' Class : No
##
We can use the following plot to determine the number of trees.
plot(rfModel)
As the number of trees increases, the error rate decreases, and then becomes almost constant. We are not able to decrease the error rate after about 100 to 200 trees. Now let’s tune our model…
t <- tuneRF(training[, -18], training[, 18], stepFactor = 0.5, plot = TRUE, ntreeTry = 200, trace = TRUE, improve = 0.05)
## mtry = 4 OOB error = 21.11%
## Searching left ...
## mtry = 8 OOB error = 21.88%
## -0.03650336 0.05
## Searching right ...
## mtry = 2 OOB error = 20.4%
## 0.03362152 0.05
In order to increase our accuracy, the graphs suggest we need to use ntrees = 200 and 2 predictors for splitting at each node (mtry)
rfModel_new <- randomForest(Churn ~., data = training, ntree = 200, mtry = 2, importance = TRUE, proximity = TRUE)
print(rfModel_new)
##
## Call:
## randomForest(formula = Churn ~ ., data = training, ntree = 200, mtry = 2, importance = TRUE, proximity = TRUE)
## Type of random forest: classification
## Number of trees: 200
## No. of variables tried at each split: 2
##
## OOB estimate of error rate: 20.52%
## Confusion matrix:
## No Yes class.error
## No 3300 322 0.08890116
## Yes 690 619 0.52711994
Error rate decreased from 21.05% to 20.5%.
pred_rf_new <- predict(rfModel_new, testing)
caret::confusionMatrix(pred_rf_new, testing$Churn2)
## Confusion Matrix and Statistics
##
## Reference
## Prediction No Yes
## No 1425 300
## Yes 127 260
##
## Accuracy : 0.7978
## 95% CI : (0.78, 0.8148)
## No Information Rate : 0.7348
## P-Value [Acc > NIR] : 9.309e-12
##
## Kappa : 0.4244
## Mcnemar's Test P-Value : < 2.2e-16
##
## Sensitivity : 0.9182
## Specificity : 0.4643
## Pos Pred Value : 0.8261
## Neg Pred Value : 0.6718
## Prevalence : 0.7348
## Detection Rate : 0.6747
## Detection Prevalence : 0.8168
## Balanced Accuracy : 0.6912
##
## 'Positive' Class : No
##
Accuracy increased to 79.8% and Sensitivity imrpoved to 90%. According to Random Forest, the most important features are the following:
varImpPlot(rfModel_new, sort=T, n.var = 10, main = 'Top 10 Feature Importance')
So, we can see that we can use Logistic Regression, Decision Trees and Random Forest for customer churn analysis for this particular dataset equally fine. From the three methods, we can conclude that tenure_group, Contract, PaperlessBilling, MonthlyCharges and InternetService appear to play a major role in customer churn.