Introduction to SuperML

SuperML R package is designed to unify the model training process in R like Python. Generally, it’s seen that people spend lot of time in searching for packages, figuring out the syntax for training machine learning models in R. This behaviour is highly apparent in users who frequently switch between R and Python. This package provides a python´s scikit-learn interface (fit, predict) to train models faster.

In addition to building machine learning models, there are handy functionalities to do feature engineering

This ambitious package is my ongoing effort to help the r-community build ML models easily and faster in R.

Install

You can install latest cran version using (recommended):

install.packages("superml")

You can install the developmemt version directly from github using:

devtools::install_github("saraswatmks/superml")

Caveats on superml installation

For machine learning, superml is based on the existing R packages. Hence, while installing the package, we don’t install all the dependencies. However, while training any model, superml will automatically install the package if its not found. Still, if you want to install all dependencies at once, you can simply do:

install.packages("superml", dependencies=TRUE)

Examples - Machine Learning Models

This package uses existing r-packages to build machine learning model. In this tutorial, we’ll use data.table R package to do all tasks related to data manipulation.

Regression Data

We’ll quickly prepare the data set to be ready to served for model training.

load("../data/reg_train.rda")
# if the above doesn't work, you can try: load("reg_train.rda")
# superml::check_package("caret")
library(data.table)
library(caret)
#> Loading required package: ggplot2
#> Loading required package: lattice
library(superml)

library(Metrics)
#> 
#> Attaching package: 'Metrics'
#> The following objects are masked from 'package:caret':
#> 
#>     precision, recall

head(reg_train)
#>       Id MSSubClass MSZoning LotFrontage LotArea Street  Alley LotShape
#>    <int>      <int>   <char>       <int>   <int> <char> <char>   <char>
#> 1:     1         60       RL          65    8450   Pave   <NA>      Reg
#> 2:     2         20       RL          80    9600   Pave   <NA>      Reg
#> 3:     3         60       RL          68   11250   Pave   <NA>      IR1
#> 4:     4         70       RL          60    9550   Pave   <NA>      IR1
#> 5:     5         60       RL          84   14260   Pave   <NA>      IR1
#> 6:     6         50       RL          85   14115   Pave   <NA>      IR1
#>    LandContour Utilities LotConfig LandSlope Neighborhood Condition1 Condition2
#>         <char>    <char>    <char>    <char>       <char>     <char>     <char>
#> 1:         Lvl    AllPub    Inside       Gtl      CollgCr       Norm       Norm
#> 2:         Lvl    AllPub       FR2       Gtl      Veenker      Feedr       Norm
#> 3:         Lvl    AllPub    Inside       Gtl      CollgCr       Norm       Norm
#> 4:         Lvl    AllPub    Corner       Gtl      Crawfor       Norm       Norm
#> 5:         Lvl    AllPub       FR2       Gtl      NoRidge       Norm       Norm
#> 6:         Lvl    AllPub    Inside       Gtl      Mitchel       Norm       Norm
#>    BldgType HouseStyle OverallQual OverallCond YearBuilt YearRemodAdd RoofStyle
#>      <char>     <char>       <int>       <int>     <int>        <int>    <char>
#> 1:     1Fam     2Story           7           5      2003         2003     Gable
#> 2:     1Fam     1Story           6           8      1976         1976     Gable
#> 3:     1Fam     2Story           7           5      2001         2002     Gable
#> 4:     1Fam     2Story           7           5      1915         1970     Gable
#> 5:     1Fam     2Story           8           5      2000         2000     Gable
#> 6:     1Fam     1.5Fin           5           5      1993         1995     Gable
#>    RoofMatl Exterior1st Exterior2nd MasVnrType MasVnrArea ExterQual ExterCond
#>      <char>      <char>      <char>     <char>      <int>    <char>    <char>
#> 1:  CompShg     VinylSd     VinylSd    BrkFace        196        Gd        TA
#> 2:  CompShg     MetalSd     MetalSd       None          0        TA        TA
#> 3:  CompShg     VinylSd     VinylSd    BrkFace        162        Gd        TA
#> 4:  CompShg     Wd Sdng     Wd Shng       None          0        TA        TA
#> 5:  CompShg     VinylSd     VinylSd    BrkFace        350        Gd        TA
#> 6:  CompShg     VinylSd     VinylSd       None          0        TA        TA
#>    Foundation BsmtQual BsmtCond BsmtExposure BsmtFinType1 BsmtFinSF1
#>        <char>   <char>   <char>       <char>       <char>      <int>
#> 1:      PConc       Gd       TA           No          GLQ        706
#> 2:     CBlock       Gd       TA           Gd          ALQ        978
#> 3:      PConc       Gd       TA           Mn          GLQ        486
#> 4:     BrkTil       TA       Gd           No          ALQ        216
#> 5:      PConc       Gd       TA           Av          GLQ        655
#> 6:       Wood       Gd       TA           No          GLQ        732
#>    BsmtFinType2 BsmtFinSF2 BsmtUnfSF TotalBsmtSF Heating HeatingQC CentralAir
#>          <char>      <int>     <int>       <int>  <char>    <char>     <char>
#> 1:          Unf          0       150         856    GasA        Ex          Y
#> 2:          Unf          0       284        1262    GasA        Ex          Y
#> 3:          Unf          0       434         920    GasA        Ex          Y
#> 4:          Unf          0       540         756    GasA        Gd          Y
#> 5:          Unf          0       490        1145    GasA        Ex          Y
#> 6:          Unf          0        64         796    GasA        Ex          Y
#>    Electrical 1stFlrSF 2ndFlrSF LowQualFinSF GrLivArea BsmtFullBath
#>        <char>    <int>    <int>        <int>     <int>        <int>
#> 1:      SBrkr      856      854            0      1710            1
#> 2:      SBrkr     1262        0            0      1262            0
#> 3:      SBrkr      920      866            0      1786            1
#> 4:      SBrkr      961      756            0      1717            1
#> 5:      SBrkr     1145     1053            0      2198            1
#> 6:      SBrkr      796      566            0      1362            1
#>    BsmtHalfBath FullBath HalfBath BedroomAbvGr KitchenAbvGr KitchenQual
#>           <int>    <int>    <int>        <int>        <int>      <char>
#> 1:            0        2        1            3            1          Gd
#> 2:            1        2        0            3            1          TA
#> 3:            0        2        1            3            1          Gd
#> 4:            0        1        0            3            1          Gd
#> 5:            0        2        1            4            1          Gd
#> 6:            0        1        1            1            1          TA
#>    TotRmsAbvGrd Functional Fireplaces FireplaceQu GarageType GarageYrBlt
#>           <int>     <char>      <int>      <char>     <char>       <int>
#> 1:            8        Typ          0        <NA>     Attchd        2003
#> 2:            6        Typ          1          TA     Attchd        1976
#> 3:            6        Typ          1          TA     Attchd        2001
#> 4:            7        Typ          1          Gd     Detchd        1998
#> 5:            9        Typ          1          TA     Attchd        2000
#> 6:            5        Typ          0        <NA>     Attchd        1993
#>    GarageFinish GarageCars GarageArea GarageQual GarageCond PavedDrive
#>          <char>      <int>      <int>     <char>     <char>     <char>
#> 1:          RFn          2        548         TA         TA          Y
#> 2:          RFn          2        460         TA         TA          Y
#> 3:          RFn          2        608         TA         TA          Y
#> 4:          Unf          3        642         TA         TA          Y
#> 5:          RFn          3        836         TA         TA          Y
#> 6:          Unf          2        480         TA         TA          Y
#>    WoodDeckSF OpenPorchSF EnclosedPorch 3SsnPorch ScreenPorch PoolArea PoolQC
#>         <int>       <int>         <int>     <int>       <int>    <int> <char>
#> 1:          0          61             0         0           0        0   <NA>
#> 2:        298           0             0         0           0        0   <NA>
#> 3:          0          42             0         0           0        0   <NA>
#> 4:          0          35           272         0           0        0   <NA>
#> 5:        192          84             0         0           0        0   <NA>
#> 6:         40          30             0       320           0        0   <NA>
#>     Fence MiscFeature MiscVal MoSold YrSold SaleType SaleCondition SalePrice
#>    <char>      <char>   <int>  <int>  <int>   <char>        <char>     <int>
#> 1:   <NA>        <NA>       0      2   2008       WD        Normal    208500
#> 2:   <NA>        <NA>       0      5   2007       WD        Normal    181500
#> 3:   <NA>        <NA>       0      9   2008       WD        Normal    223500
#> 4:   <NA>        <NA>       0      2   2006       WD       Abnorml    140000
#> 5:   <NA>        <NA>       0     12   2008       WD        Normal    250000
#> 6:  MnPrv        Shed     700     10   2009       WD        Normal    143000

split <- createDataPartition(y = reg_train$SalePrice, p = 0.7)
xtrain <- reg_train[split$Resample1]
xtest <- reg_train[!split$Resample1]
# remove features with 90% or more missing values
# we will also remove the Id column because it doesn't contain
# any useful information
na_cols <- colSums(is.na(xtrain)) / nrow(xtrain)
na_cols <- names(na_cols[which(na_cols > 0.9)])

xtrain[, c(na_cols, "Id") := NULL]
xtest[, c(na_cols, "Id") := NULL]

# encode categorical variables
cat_cols <- names(xtrain)[sapply(xtrain, is.character)]

for(c in cat_cols){
    lbl <- LabelEncoder$new()
    lbl$fit(c(xtrain[[c]], xtest[[c]]))
    xtrain[[c]] <- lbl$transform(xtrain[[c]])
    xtest[[c]] <- lbl$transform(xtest[[c]])
}
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA' 
#> The data contains NA values. Imputing NA with 'NA'

# removing noise column
noise <- c('GrLivArea','TotalBsmtSF')

xtrain[, c(noise) := NULL]
xtest[, c(noise) := NULL]

# fill missing value with  -1
xtrain[is.na(xtrain)] <- -1
xtest[is.na(xtest)] <- -1

KNN Regression

knn <- KNNTrainer$new(k = 2,prob = T,type = 'reg')
knn$fit(train = xtrain, test = xtest, y = 'SalePrice')
probs <- knn$predict(type = 'prob')
labels <- knn$predict(type='raw')
rmse(actual = xtest$SalePrice, predicted=labels)
#> [1] 52234.77

SVM Regression

svm <- SVMTrainer$new()
svm$fit(xtrain, 'SalePrice')
pred <- svm$predict(xtest)
rmse(actual = xtest$SalePrice, predicted = pred)

Simple Regresison

lf <- LMTrainer$new(family="gaussian")
lf$fit(X = xtrain, y = "SalePrice")
summary(lf$model)
#> 
#> Call:
#> stats::glm(formula = f, family = self$family, data = X, weights = self$weights)
#> 
#> Coefficients:
#>                 Estimate Std. Error t value Pr(>|t|)    
#> (Intercept)    8.944e+05  1.689e+06   0.530 0.596554    
#> MSSubClass    -1.455e+02  5.032e+01  -2.892 0.003919 ** 
#> MSZoning      -1.113e+03  1.583e+03  -0.703 0.482330    
#> LotFrontage   -3.261e+01  3.505e+01  -0.930 0.352388    
#> LotArea        3.831e-01  1.436e-01   2.668 0.007764 ** 
#> Street        -2.280e+04  1.644e+04  -1.387 0.165699    
#> LotShape       6.598e+02  2.209e+03   0.299 0.765231    
#> LandContour    4.753e+02  2.130e+03   0.223 0.823471    
#> Utilities     -5.652e+04  3.608e+04  -1.566 0.117589    
#> LotConfig      1.372e+03  1.149e+03   1.194 0.232908    
#> LandSlope      1.482e+04  5.102e+03   2.906 0.003748 ** 
#> Neighborhood  -4.745e+02  2.080e+02  -2.281 0.022747 *  
#> Condition1    -2.340e+03  9.037e+02  -2.589 0.009767 ** 
#> Condition2    -1.001e+04  2.893e+03  -3.459 0.000567 ***
#> BldgType       2.449e+02  2.257e+03   0.108 0.913631    
#> HouseStyle     4.509e+01  9.958e+02   0.045 0.963896    
#> OverallQual    1.428e+04  1.441e+03   9.903  < 2e-16 ***
#> OverallCond    6.695e+03  1.309e+03   5.113 3.84e-07 ***
#> YearBuilt      3.830e+02  8.808e+01   4.349 1.52e-05 ***
#> YearRemodAdd   9.095e+01  8.428e+01   1.079 0.280824    
#> RoofStyle      6.199e+03  2.096e+03   2.958 0.003174 ** 
#> RoofMatl      -1.560e+04  2.326e+03  -6.708 3.38e-11 ***
#> Exterior1st   -4.137e+02  7.170e+02  -0.577 0.564094    
#> Exterior2nd    7.765e+02  6.298e+02   1.233 0.217876    
#> MasVnrType     3.406e+03  1.723e+03   1.977 0.048305 *  
#> MasVnrArea     2.245e+01  7.580e+00   2.962 0.003130 ** 
#> ExterQual      7.285e+02  2.571e+03   0.283 0.776970    
#> ExterCond      1.254e+03  2.529e+03   0.496 0.620077    
#> Foundation    -3.760e+03  1.983e+03  -1.896 0.058278 .  
#> BsmtQual       8.094e+03  1.789e+03   4.525 6.81e-06 ***
#> BsmtCond      -1.099e+03  2.544e+03  -0.432 0.665845    
#> BsmtExposure   1.243e+03  1.127e+03   1.103 0.270437    
#> BsmtFinType1  -1.561e+03  9.268e+02  -1.685 0.092409 .  
#> BsmtFinSF1     8.764e+00  6.632e+00   1.322 0.186650    
#> BsmtFinType2  -1.338e+03  1.397e+03  -0.958 0.338329    
#> BsmtFinSF2     1.867e+01  9.873e+00   1.891 0.058874 .  
#> BsmtUnfSF      2.562e+00  6.476e+00   0.396 0.692462    
#> Heating       -1.378e+02  3.972e+03  -0.035 0.972333    
#> HeatingQC     -2.959e+03  1.508e+03  -1.962 0.050054 .  
#> CentralAir     5.970e+03  5.599e+03   1.066 0.286565    
#> Electrical     1.997e+03  2.182e+03   0.915 0.360373    
#> `1stFlrSF`     5.660e+01  7.848e+00   7.212 1.13e-12 ***
#> `2ndFlrSF`     5.686e+01  6.063e+00   9.378  < 2e-16 ***
#> LowQualFinSF   3.005e+01  2.404e+01   1.250 0.211451    
#> BsmtFullBath   1.040e+04  3.130e+03   3.324 0.000922 ***
#> BsmtHalfBath   4.035e+03  4.927e+03   0.819 0.413038    
#> FullBath       9.275e+03  3.358e+03   2.762 0.005859 ** 
#> HalfBath      -1.313e+03  3.215e+03  -0.408 0.683056    
#> BedroomAbvGr  -6.794e+03  2.064e+03  -3.291 0.001034 ** 
#> KitchenAbvGr  -2.080e+04  7.048e+03  -2.951 0.003245 ** 
#> KitchenQual    9.268e+03  1.702e+03   5.446 6.58e-08 ***
#> TotRmsAbvGrd   1.536e+03  1.505e+03   1.021 0.307646    
#> Functional    -4.123e+03  1.579e+03  -2.611 0.009165 ** 
#> Fireplaces    -8.830e+02  2.784e+03  -0.317 0.751201    
#> FireplaceQu    4.331e+03  1.516e+03   2.856 0.004378 ** 
#> GarageType    -7.373e+02  1.304e+03  -0.565 0.571969    
#> GarageYrBlt   -8.845e+00  5.892e+00  -1.501 0.133621    
#> GarageFinish   9.045e+02  1.571e+03   0.576 0.564916    
#> GarageCars     1.663e+04  3.630e+03   4.583 5.20e-06 ***
#> GarageArea    -8.641e+00  1.170e+01  -0.739 0.460278    
#> GarageQual     5.039e+03  4.076e+03   1.236 0.216717    
#> GarageCond    -4.130e+03  3.990e+03  -1.035 0.300889    
#> PavedDrive    -2.232e+03  3.608e+03  -0.619 0.536311    
#> WoodDeckSF     2.317e+01  9.599e+00   2.413 0.015991 *  
#> OpenPorchSF    5.149e+00  1.874e+01   0.275 0.783614    
#> EnclosedPorch -4.094e+00  1.910e+01  -0.214 0.830312    
#> `3SsnPorch`    3.062e+01  3.466e+01   0.884 0.377084    
#> ScreenPorch    4.949e+01  2.121e+01   2.333 0.019862 *  
#> PoolArea      -3.789e+01  2.823e+01  -1.342 0.179764    
#> Fence         -2.077e+03  1.398e+03  -1.486 0.137708    
#> MiscVal        3.181e+00  3.688e+00   0.863 0.388567    
#> MoSold        -1.435e+02  4.089e+02  -0.351 0.725680    
#> YrSold        -9.315e+02  8.418e+02  -1.107 0.268748    
#> SaleType       3.539e+03  1.561e+03   2.268 0.023580 *  
#> SaleCondition -6.941e+02  1.484e+03  -0.468 0.640153    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> (Dispersion parameter for gaussian family taken to be 1115076131)
#> 
#>     Null deviance: 6.6565e+12  on 1023  degrees of freedom
#> Residual deviance: 1.0582e+12  on  949  degrees of freedom
#> AIC: 24312
#> 
#> Number of Fisher Scoring iterations: 2
predictions <- lf$predict(df = xtest)
rmse(actual = xtest$SalePrice, predicted = predictions)
#> [1] 28861.82

Lasso Regression

lf <- LMTrainer$new(family = "gaussian", alpha = 1, lambda = 1000)
lf$fit(X = xtrain, y = "SalePrice")
predictions <- lf$predict(df = xtest)
rmse(actual = xtest$SalePrice, predicted = predictions)
#> [1] 34580.86

Ridge Regression

lf <- LMTrainer$new(family = "gaussian", alpha=0)
lf$fit(X = xtrain, y = "SalePrice")
predictions <- lf$predict(df = xtest)
rmse(actual = xtest$SalePrice, predicted = predictions)
#> [1] 34669.58

Logistic Regression with CV

lf <- LMTrainer$new(family = "gaussian")
lf$cv_model(X = xtrain, y = 'SalePrice', nfolds = 5, parallel = FALSE)
predictions <- lf$cv_predict(df = xtest)
coefs <- lf$get_importance()
rmse(actual = xtest$SalePrice, predicted = predictions)

Random Forest

rf <- RFTrainer$new(n_estimators = 500,classification = 0)
rf$fit(X = xtrain, y = "SalePrice")
pred <- rf$predict(df = xtest)
rf$get_importance()
#>               tmp.order.tmp..decreasing...TRUE..
#> OverallQual                         814703838356
#> GarageCars                          508099761498
#> 1stFlrSF                            491734847096
#> GarageArea                          451112100987
#> YearBuilt                           352619308205
#> FullBath                            310264362110
#> BsmtFinSF1                          299185716539
#> KitchenQual                         259901998865
#> GarageYrBlt                         248486922461
#> 2ndFlrSF                            247859932853
#> TotRmsAbvGrd                        190330252732
#> LotArea                             177406375972
#> ExterQual                           174207748492
#> YearRemodAdd                        154261461583
#> Fireplaces                          132043881664
#> FireplaceQu                         131295258466
#> MasVnrArea                          119617322793
#> BsmtQual                            113566441087
#> Foundation                          104858549549
#> OpenPorchSF                         103666794521
#> LotFrontage                          94344420430
#> BsmtUnfSF                            74477805402
#> Neighborhood                         73691766244
#> BsmtFinType1                         73639055397
#> WoodDeckSF                           66183677474
#> BedroomAbvGr                         59485192985
#> GarageType                           52019228588
#> HeatingQC                            51010976404
#> Exterior2nd                          44813371442
#> RoofStyle                            41276355532
#> MoSold                               37891677345
#> MSSubClass                           36326975984
#> OverallCond                          35379904577
#> HouseStyle                           33748494732
#> BsmtFullBath                         31504803491
#> HalfBath                             30594380307
#> Exterior1st                          30012907766
#> GarageFinish                         26533940995
#> BsmtExposure                         22679562631
#> SaleCondition                        22373273806
#> LotShape                             20682865510
#> PoolArea                             19708390441
#> MSZoning                             19475426262
#> YrSold                               19240548414
#> CentralAir                           17744449654
#> ScreenPorch                          16424157765
#> LotConfig                            16421910403
#> MasVnrType                           16011429349
#> SaleType                             15582285428
#> BsmtHalfBath                         14732371130
#> LandContour                          14474438364
#> BldgType                             13815125878
#> RoofMatl                             13760138829
#> BsmtFinSF2                           13604724855
#> Fence                                13454832727
#> LandSlope                            12332196861
#> Condition1                           11120609995
#> GarageCond                           10972277588
#> EnclosedPorch                         9643303512
#> GarageQual                            9415428695
#> BsmtCond                              9382636762
#> BsmtFinType2                          8533999374
#> Functional                            6159744474
#> PavedDrive                            5450372868
#> ExterCond                             5352738731
#> KitchenAbvGr                          5153125554
#> Electrical                            3977319783
#> Condition2                            2664097220
#> Heating                               2321597300
#> 3SsnPorch                             2036587090
#> MiscVal                               1623603919
#> LowQualFinSF                          1090567307
#> Street                                 541659531
#> Utilities                               21305800
rmse(actual = xtest$SalePrice, predicted = pred)
#> [1] 25446.54

Xgboost

xgb <- XGBTrainer$new(objective = "reg:linear"
                      , n_estimators = 500
                      , eval_metric = "rmse"
                      , maximize = F
                      , learning_rate = 0.1
                      ,max_depth = 6)
xgb$fit(X = xtrain, y = "SalePrice", valid = xtest)
pred <- xgb$predict(xtest)
rmse(actual = xtest$SalePrice, predicted = pred)

Grid Search

xgb <- XGBTrainer$new(objective = "reg:linear")

gst <- GridSearchCV$new(trainer = xgb,
                             parameters = list(n_estimators = c(10,50), max_depth = c(5,2)),
                             n_folds = 3,
                             scoring = c('accuracy','auc'))
gst$fit(xtrain, "SalePrice")
gst$best_iteration()

Random Search

rf <- RFTrainer$new()
rst <- RandomSearchCV$new(trainer = rf,
                             parameters = list(n_estimators = c(5,10),
                             max_depth = c(5,2)),
                             n_folds = 3,
                             scoring = c('accuracy','auc'),
                             n_iter = 3)
rst$fit(xtrain, "SalePrice")
#> [1] "In total, 3 models will be trained"
rst$best_iteration()
#> $n_estimators
#> [1] 5
#> 
#> $max_depth
#> [1] 5
#> 
#> $accuracy_avg
#> [1] 0.01269642
#> 
#> $accuracy_sd
#> [1] 0.001703672
#> 
#> $auc_avg
#> [1] NaN
#> 
#> $auc_sd
#> [1] NA

Binary Classification Data

Here, we will solve a simple binary classification problem (predict people who survived on titanic ship). The idea here is to demonstrate how to use this package to solve classification problems.

Data Preparation

# load class
load('../data/cla_train.rda')
# if the above doesn't work, you can try: load("cla_train.rda")

head(cla_train)
#>    PassengerId Survived Pclass
#>          <int>    <int>  <int>
#> 1:           1        0      3
#> 2:           2        1      1
#> 3:           3        1      3
#> 4:           4        1      1
#> 5:           5        0      3
#> 6:           6        0      3
#>                                                   Name    Sex   Age SibSp Parch
#>                                                 <char> <char> <num> <int> <int>
#> 1:                             Braund, Mr. Owen Harris   male    22     1     0
#> 2: Cumings, Mrs. John Bradley (Florence Briggs Thayer) female    38     1     0
#> 3:                              Heikkinen, Miss. Laina female    26     0     0
#> 4:        Futrelle, Mrs. Jacques Heath (Lily May Peel) female    35     1     0
#> 5:                            Allen, Mr. William Henry   male    35     0     0
#> 6:                                    Moran, Mr. James   male    NA     0     0
#>              Ticket    Fare  Cabin Embarked
#>              <char>   <num> <char>   <char>
#> 1:        A/5 21171  7.2500               S
#> 2:         PC 17599 71.2833    C85        C
#> 3: STON/O2. 3101282  7.9250               S
#> 4:           113803 53.1000   C123        S
#> 5:           373450  8.0500               S
#> 6:           330877  8.4583               Q

# split the data
split <- createDataPartition(y = cla_train$Survived,p = 0.7)
xtrain <- cla_train[split$Resample1]
xtest <- cla_train[!split$Resample1]

# encode categorical variables - shorter way
for(c in c('Embarked','Sex','Cabin')) {
    lbl <- LabelEncoder$new()
    lbl$fit(c(xtrain[[c]], xtest[[c]]))
    xtrain[[c]] <- lbl$transform(xtrain[[c]])
    xtest[[c]] <- lbl$transform(xtest[[c]])
}
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA' 
#> The data contains blank values. Imputing them with 'NA'

# impute missing values
xtrain[, Age := replace(Age, is.na(Age), median(Age, na.rm = T))]
xtest[, Age := replace(Age, is.na(Age), median(Age, na.rm = T))]

# drop these features
to_drop <- c('PassengerId','Ticket','Name')

xtrain <- xtrain[,-c(to_drop), with=F]
xtest <- xtest[,-c(to_drop), with=F]

Now, our data is ready to be served for model training. Let’s do it.

KNN Classification

knn <- KNNTrainer$new(k = 2,prob = T,type = 'class')
knn$fit(train = xtrain, test = xtest, y = 'Survived')
probs <- knn$predict(type = 'prob')
labels <- knn$predict(type = 'raw')
auc(actual = xtest$Survived, predicted = labels)
#> [1] 0.6385027

Naive Bayes Classification

nb <- NBTrainer$new()
nb$fit(xtrain, 'Survived')
pred <- nb$predict(xtest)
#> Warning: predict.naive_bayes(): more features in the newdata are provided as
#> there are probability tables in the object. Calculation is performed based on
#> features to be found in the tables.
auc(actual = xtest$Survived, predicted = pred)
#> [1] 0.7771836

SVM Classification

#predicts labels
svm <- SVMTrainer$new()
svm$fit(xtrain, 'Survived')
pred <- svm$predict(xtest)
auc(actual = xtest$Survived, predicted=pred)

Logistic Regression

lf <- LMTrainer$new(family = "binomial")
lf$fit(X = xtrain, y = "Survived")
summary(lf$model)
#> 
#> Call:
#> stats::glm(formula = f, family = self$family, data = X, weights = self$weights)
#> 
#> Coefficients:
#>              Estimate Std. Error z value Pr(>|z|)    
#> (Intercept)  1.830070   0.616894   2.967  0.00301 ** 
#> Pclass      -0.980785   0.192493  -5.095 3.48e-07 ***
#> Sex          2.508241   0.230374  10.888  < 2e-16 ***
#> Age         -0.041034   0.009309  -4.408 1.04e-05 ***
#> SibSp       -0.235520   0.117715  -2.001  0.04542 *  
#> Parch       -0.098742   0.137791  -0.717  0.47361    
#> Fare         0.001281   0.002842   0.451  0.65230    
#> Cabin        0.008408   0.004786   1.757  0.07899 .  
#> Embarked     0.248088   0.166616   1.489  0.13649    
#> ---
#> Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
#> 
#> (Dispersion parameter for binomial family taken to be 1)
#> 
#>     Null deviance: 831.52  on 623  degrees of freedom
#> Residual deviance: 564.76  on 615  degrees of freedom
#> AIC: 582.76
#> 
#> Number of Fisher Scoring iterations: 5
predictions <- lf$predict(df = xtest)
auc(actual = xtest$Survived, predicted = predictions)
#> [1] 0.8832145

Lasso Logistic Regression

lf <- LMTrainer$new(family="binomial", alpha=1)
lf$cv_model(X = xtrain, y = "Survived", nfolds = 5, parallel = FALSE)
pred <- lf$cv_predict(df = xtest)
auc(actual = xtest$Survived, predicted = pred)

Ridge Logistic Regression

lf <- LMTrainer$new(family="binomial", alpha=0)
lf$cv_model(X = xtrain, y = "Survived", nfolds = 5, parallel = FALSE)
pred <- lf$cv_predict(df = xtest)
auc(actual = xtest$Survived, predicted = pred)

Random Forest

rf <- RFTrainer$new(n_estimators = 500,classification = 1, max_features = 3)
rf$fit(X = xtrain, y = "Survived")

pred <- rf$predict(df = xtest)
rf$get_importance()
#>          tmp.order.tmp..decreasing...TRUE..
#> Sex                                69.10742
#> Fare                               57.96084
#> Age                                48.50156
#> Pclass                             23.91175
#> Cabin                              21.19329
#> SibSp                              12.58503
#> Parch                              10.55128
#> Embarked                           10.07059

auc(actual = xtest$Survived, predicted = pred)
#> [1] 0.7988414

Xgboost

xgb <- XGBTrainer$new(objective = "binary:logistic"
                      , n_estimators = 500
                      , eval_metric = "auc"
                      , maximize = T
                      , learning_rate = 0.1
                      ,max_depth = 6)
xgb$fit(X = xtrain, y = "Survived", valid = xtest)

pred <- xgb$predict(xtest)
auc(actual = xtest$Survived, predicted = pred)

Grid Search

xgb <- XGBTrainer$new(objective="binary:logistic")
gst <-GridSearchCV$new(trainer = xgb,
                             parameters = list(n_estimators = c(10,50),
                             max_depth = c(5,2)),
                             n_folds = 3,
                             scoring = c('accuracy','auc'))
gst$fit(xtrain, "Survived")
gst$best_iteration()

Random Search

rf <- RFTrainer$new()
rst <- RandomSearchCV$new(trainer = rf,
                             parameters = list(n_estimators = c(10,50), max_depth = c(5,2)),
                             n_folds = 3,
                             scoring = c('accuracy','auc'),
                             n_iter = 3)
rst$fit(xtrain, "Survived")
#> [1] "In total, 3 models will be trained"
rst$best_iteration()
#> $n_estimators
#> [1] 50
#> 
#> $max_depth
#> [1] 5
#> 
#> $accuracy_avg
#> [1] 0.8028846
#> 
#> $accuracy_sd
#> [1] 0.01733438
#> 
#> $auc_avg
#> [1] 0.7804264
#> 
#> $auc_sd
#> [1] 0.02631447

Let’s create some new feature based on target variable using target encoding and test a model.

# add target encoding features
xtrain[, feat_01 := smoothMean(train_df = xtrain,
                        test_df = xtest,
                        colname = "Embarked",
                        target = "Survived")$train[[2]]]
xtest[, feat_01 := smoothMean(train_df = xtrain,
                               test_df = xtest,
                               colname = "Embarked",
                               target = "Survived")$test[[2]]]

# train a random forest
# Random Forest
rf <- RFTrainer$new(n_estimators = 500,classification = 1, max_features = 4)
rf$fit(X = xtrain, y = "Survived")
pred <- rf$predict(df = xtest)
rf$get_importance()
#>          tmp.order.tmp..decreasing...TRUE..
#> Sex                               71.417138
#> Fare                              61.039958
#> Age                               51.787990
#> Pclass                            24.257112
#> Cabin                             21.549374
#> SibSp                             12.374317
#> Parch                             10.392826
#> feat_01                            6.490151
#> Embarked                           6.270997

auc(actual = xtest$Survived, predicted = pred)
#> [1] 0.7988414