Data Scientist Course in Calicut

Data Science project management methodology, CRISP-DM will be explained in this module in finer detail. Learn about Data Collection, Data Cleansing, Data Preparation, Data Munging, Data Wrapping, etc. Learn about the preliminary steps taken to churn the data, known as exploratory data analysis. In this module, you also are introduced to statistical calculations which are used to derive information from data. We will begin to understand how to perform a descriptive analysis.

• Machine Learning project management methodology
• Data Collection - Surveys and Design of Experiments
• Data Types namely Continuous, Discrete, Categorical, Count, Qualitative, Quantitative and its identification and application
• Further classification of data in terms of Nominal, Ordinal, Interval & Ratio types
• Balanced versus Imbalanced datasets
• Cross Sectional versus Time Series vs Panel / Longitudinal Data
• Batch Processing vs Real Time Processing
• Structured versus Unstructured vs Semi-Structured Data
• Big vs Not-Big Data
• Data Cleaning / Preparation - Outlier Analysis, Missing Values Imputation Techniques, Transformations, Normalization / Standardization, Discretization
• Sampling techniques for handling Balanced vs. Imbalanced Datasets
• What is the Sampling Funnel and its application and its components?
  • Population
  • Sampling frame
  • Simple random sampling
  • Sample
• Measures of Central Tendency & Dispersion
  • Population
  • Mean/Average, Median, Mode
  • Variance, Standard Deviation, Range

Learn about various statistical calculations used to capture business moments for enabling decision makers to make data driven decisions. You will learn about the distribution of the data and its shape using these calculations. Understand to intercept information by representing data by visuals. Also learn about Univariate analysis, Bivariate analysis and Multivariate analysis.

• Measure of Skewness
• Measure of Kurtosis
• Spread of the Data
• Various graphical techniques to understand data
  • Bar Plot
  • Histogram
  • Boxplot
  • Scatter Plot

Data Visualization helps understand the patterns or anomalies in the data easily and learn about various graphical representations in this module. Understand the terms univariate and bivariate and the plots used to analyze in 2D dimensions. Understand how to derive conclusions on business problems using calculations performed on sample data. You will learn the concepts to deal with the variations that arise while analyzing different samples for the same population using the central limit theorem.

• Line Chart
• Pair Plot
• Sample Statistics
• Population Parameters
• Inferential Statistics

In this tutorial you will learn in detail about continuous probability distribution. Understand the properties of a continuous random variable and its distribution under normal conditions. To identify the properties of a continuous random variable, statisticians have defined a variable as a standard, learning the properties of the standard variable and its distribution. You will learn to check if a continuous random variable is following normal distribution using a normal Q-Q plot. Learn the science behind the estimation of value for a population using sample data.

• Random Variable and its definition
• Probability & Probability Distribution
  • Continuous Probability Distribution / Probability Density Function
  • Discrete Probability Distribution / Probability Mass Function
• Normal Distribution
• Standard Normal Distribution / Z distribution
• Z scores and the Z table
• QQ Plot / Quantile - Quantile plot
• Sampling Variation
• Central Limit Theorem
• Sample size calculator
• Confidence interval - concept
• Confidence interval with sigma
• T-distribution / Student's-t distribution
• Confidence interval
  • Population parameter with Standard deviation known
  • Population parameter with Standard deviation not known
• A complete recap of Statistics

Learn to frame business statements by making assumptions. Understand how to perform testing of these assumptions to make decisions for business problems. Learn about different types of Hypothesis testing and its statistics. You will learn the different conditions of the Hypothesis table, namely Null Hypothesis, Alternative hypothesis, Type I error and Type II error. The prerequisites for conducting a Hypothesis test, interpretation of the results will be discussed in this module.

• Formulating a Hypothesis
• Choosing Null and Alternative Hypothesis
• Type I or Alpha Error and Type II or Beta Error
• Confidence Level, Significance Level, Power of Test
• Comparative study of sample proportions using Hypothesis testing
• 2 Sample t-test
• ANOVA
• 2 Proportion test
• Chi-Square test

Data Mining supervised learning is all about making predictions for an unknown dependent variable using mathematical equations explaining the relationship with independent variables. Revisit the school math with the equation of a straight line. Learn about the components of Linear Regression with the equation of the regression line. Get introduced to Linear Regression analysis with a use case for prediction of a continuous dependent variable. Understand about ordinary least squares technique.

• Scatter diagram
  • Correlation analysis
  • Correlation coefficient
• Ordinary least squares
• Principles of regression
• Simple Linear Regression
• Exponential Regression, Logarithmic Regression, Quadratic or Polynomial Regression
• Confidence Interval versus Prediction Interval
• Heteroscedasticity / Equal Variance

In the continuation to Regression analysis study you will learn how to deal with multiple independent variables affecting the dependent variable. Learn about the conditions and assumptions to perform linear regression analysis and the workarounds used to follow the conditions. Understand the steps required to perform the evaluation of the model and to improvise the prediction accuracies. You will be introduced to concepts of variance and bias.

• LINE assumption
  • Linearity
  • Independence
  • Normality
  • Equal Variance / Homoscedasticity
• Collinearity (Variance Inflation Factor)
• Multiple Linear Regression
• Model Quality metrics
• Deletion Diagnostics

Learn about overfitting and underfitting conditions for prediction models developed. We need to strike the right balance between overfitting and underfitting, learn about regularization techniques L1 norm and L2 norm used to reduce these abnormal conditions. The regression techniques Lasso and Ridge techniques are discussed in this module .

• Understanding Overfitting (Variance) vs. Underfitting (Bias)
• Generalization error and Regularization techniques
• Different Error functions or Loss functions or Cost functions
• Lasso Regression
• Ridge Regression

You have learnt about predicting a continuous dependent variable. As part of this module, you will continue to learn Regression techniques applied to predict attribute Data. Learn about the principles of the logistic regression model, understand the sigmoid curve, the usage of cutoff value to interpret the probable outcome of the logistic regression model. Learn about the confusion matrix and its parameters to evaluate the outcome of the prediction model. Also, learn about maximum likelihood estimation.

• Principles of Logistic regression
• Types of Logistic regression
• Assumption & Steps in Logistic regression
• Analysis of Simple logistic regression results
• Multiple Logistic regression
• Confusion matrix
  • False Positive, False Negative
  • True Positive, True Negative
  • Sensitivity, Recall, Specificity, F1
• Receiver operating characteristics curve (ROC curve)
• Precision Recall (P-R) curve
• Lift charts and Gain charts

Extension to logistic regression We have a multinomial regression technique used to predict a multiple categorical outcome. Understand the concept of multi logit equations, baseline and making classifications using probability outcomes. Learn about handling multiple categories in output variables including nominal as well as ordinal data.

• Logit and Log-Likelihood
• Category Baselining
• Modeling Nominal categorical data
• Handling Ordinal Categorical Data
• Interpreting the results of coefficient values

As part of this module you learn further different regression techniques used for predicting discrete data. These regression techniques are used to analyze the numeric data known as count data. Based on the discrete probability distributions namely Poisson, negative binomial distribution the regression models try to fit the data to these distributions. Alternatively, when excessive zeros exist in the dependent variable, zero-inflated models are preferred, you will learn the types of zero-inflated models used to fit excessive zeros data.

• Poisson Regression
• Poisson Regression with Offset
• Negative Binomial Regression
• Treatment of data with Excessive Zeros
  • Zero-inflated Poisson
  • Zero-inflated Negative Binomial
  • Hurdle Model

Data mining unsupervised techniques are used as EDA techniques to derive insights from the business data. In this first module of unsupervised learning, get introduced to clustering algorithms. Learn about different approaches for data segregation to create homogeneous groups of data. Hierarchical clustering, K means clustering are most commonly used clustering algorithms. Understand the different mathematical approaches to perform data segregation. Also learn about variations in K-means clustering like K-medoids, K-mode techniques, learn to handle large data sets using CLARA technique.

• • Hierarchical • Supervised vs Unsupervised learning • Data Mining Process • Hierarchical Clustering / Agglomerative Clustering • Dendrogram • Measure of distance
  • Numeric
    • Euclidean, Manhattan, Mahalanobis
  • Categorical
    • Binary Euclidean
    • Simple Matching Coefficient
    • Jaquard's Coefficient
  • Mixed
    • Gower's General Dissimilarity Coefficient
  • Types of Linkages
    • Single Linkage / Nearest Neighbour
    • Complete Linkage / Farthest Neighbour
    • Average Linkage
    • Centroid Linkage
  • K-Means Clustering
    • Measurement metrics of clustering
      • Within the Sum of Squares
      • Between the Sum of Squares
      • Total Sum of Squares
    • Choosing the ideal K value using Scree Plot / Elbow Curve
    • Other Clustering Techniques
      • K-Medians
      • K-Medoids
      • K-Modes
      • Clustering Large Application (CLARA)
      • Partitioning Around Medoids (PAM)
      • Density-based spatial clustering of applications with noise (DBSCAN)

Dimension Reduction (PCA) / Factor Analysis Description: Learn to handle high dimensional data. The performance will be hit when the data has a high number of dimensions and machine learning techniques training becomes very complex, as part of this module you will learn to apply data reduction techniques without any variable deletion. Learn the advantages of dimensional reduction techniques. Also, learn about yet another technique called Factor Analysis.

• Why Dimension Reduction
• Advantages of PCA
• Calculation of PCA weights
• 2D Visualization using Principal components
• Basics of Matrix Algebra
• Factor Analysis

Learn to measure the relationship between entities. Bundle offers are defined based on this measure of dependency between products. Understand the metrics Support, Confidence and Lift used to define the rules with the help of Apriori algorithm. Learn pros and cons of each of the metrics used in Association rules.

• What is Market Basket / Affinity Analysis
• Measure of Association
  • Support
  • Confidence
  • Lift Ratio
• Apriori Algorithm
• Sequential Pattern Mining

Personalized recommendations made in e-commerce are based on all the previous transactions made. Learn the science of making these recommendations using measuring similarity between customers. The various methods applied for collaborative filtering, their pros and cons, SVD method used for recommendations of movies by Netflix will be discussed as part of this module.

• User-based Collaborative Filtering
• A measure of distance/similarity between users
• Driver for Recommendation
• Computation Reduction Techniques
• Search based methods/Item to Item Collaborative Filtering
• SVD in recommendation
• The vulnerability of recommendation systems

Study of a network with quantifiable values is known as network analytics. The vertex and edge are the node and connection of a network, learn about the statistics used to calculate the value of each node in the network. You will also learn about the google page ranking algorithm as part of this module.

• Definition of a network (the LinkedIn analogy)
• The measure of Node strength in a Network
  • Degree centrality
  • Closeness centrality
  • Eigenvector centrality
  • Adjacency matrix
  • Betweenness centrality
  • Cluster coefficient
• Introduction to Google page ranking

k Nearest Neighbor algorithm is distance based machine learning algorithm. Learn to classify the dependent variable using the appropriate k value. The k-NN classifier also known as lazy learner is a very popular algorithm and one of the easiest for application.

• Deciding the K value
• Thumb rule in choosing the K value
• Building a KNN model by splitting the data
• Checking for Underfitting and Overfitting in KNN
• Generalization and Regulation Techniques to avoid overfitting in KNN

Decision Tree & Random forest are some of the most powerful classifier algorithms based on classification rules. In this tutorial, you will learn about deriving the rules for classifying the dependent variable by constructing the best tree using statistical measures to capture the information from each of the attributes. Random forest is an ensemble technique constructed using multiple Decision trees and the final outcome is drawn from the aggregating the results obtained from these combinations of trees.

• Elements of classification tree - Root node, Child Node, Leaf Node, etc.
• Greedy algorithm
• Measure of Entropy
• Attribute selection using Information gain
• Ensemble techniques - Stacking, Boosting and Bagging
• Decision Tree C5.0 and understanding various arguments
• Checking for Underfitting and Overfitting in Decision Tree
• Generalization and Regulation Techniques to avoid overfitting in Decision Tree
• Random Forest and understanding various arguments
• Checking for Underfitting and Overfitting in Random Forest
• Generalization and Regulation Techniques to avoid overfitting in Random Forest

Learn about improving reliability and accuracy of decision tree models using ensemble techniques. Bagging and Boosting are the go to techniques in ensemble techniques. The parallel and sequential approaches taken in Bagging and Boosting methods are discussed in this module.

• Overfitting
• Underfitting
• Pruning
• Boosting
• Bagging or Bootstrap aggregating

The Boosting algorithms AdaBoost and Extreme Gradient Boosting are discussed as part of this continuation module You will also learn about stacking methods. Learn about these algorithms which are providing unprecedented accuracy and helping many aspiring data scientists win the first place in various competitions such as Kaggle, CrowdAnalytix, etc.

• AdaBoost / Adaptive Boosting Algorithm
• Checking for Underfitting and Overfitting in AdaBoost
• Generalization and Regulation Techniques to avoid overfitting in AdaBoost
• Gradient Boosting Algorithm
• Checking for Underfitting and Overfitting in Gradient Boosting
• Generalization and Regulation Techniques to avoid overfitting in Gradient Boosting
• Extreme Gradient Boosting (XGB) Algorithm
• Checking for Underfitting and Overfitting in XGB
• Generalization and Regulation Techniques to avoid overfitting in XGB

Learn to analyse the unstructured textual data to derive meaningful insights. Understand the language quirks to perform data cleansing, extract features using a bag of words and construct the key-value pair matrix called DTM. Learn to understand the sentiment of customers from their feedback to take appropriate actions. Advanced concepts of text mining will also be discussed which help to interpret the context of the raw text data. Topic models using LDA algorithm, emotion mining using lexicons are discussed as part of NLP module.

• Sources of data
• Bag of words
• Pre-processing, corpus Document Term Matrix (DTM) & TDM
• Word Clouds
• Corpus level word clouds
  • Sentiment Analysis
  • Positive Word clouds
  • Negative word clouds
  • Unigram, Bigram, Trigram
• Semantic network
• Clustering
• Extract user reviews of the product/services from Amazon, Snapdeal and trip advisor
• Install Libraries from Shell
• Extraction and text analytics in Python
• LDA / Latent Dirichlet Allocation
• Topic Modelling
• Sentiment Extraction
• Lexicons & Emotion Mining

Revise Bayes theorem to develop a classification technique for Machine learning. In this tutorial you will learn about joint probability and its applications. Learn how to predict whether an incoming email is a spam or a ham email. Learn about Bayesian probability and the applications in solving complex business problems.

• Probability – Recap
• Bayes Rule
• Naïve Bayes Classifier
• Text Classification using Naive Bayes
• Checking for Underfitting and Overfitting in Naive Bayes
• Generalization and Regulation Techniques to avoid overfitting in Naive Bayes

Perceptron algorithm is defined based on a biological brain model. You will talk about the parameters used in the perceptron algorithm which is the foundation of developing much complex neural network models for AI applications. Understand the application of perceptron algorithms to classify binary data in a linearly separable scenario.

• Neurons of a Biological Brain
• Artificial Neuron
• Perceptron
• Perceptron Algorithm
• Use case to classify a linearly separable data
• Multilayer Perceptron to handle non-linear data

Neural Network is a black box technique used for deep learning models. Learn the logic of training and weights calculations using various parameters and their tuning. Understand the activation function and integration functions used in developing a neural network.

• Integration functions
• Activation functions
• Weights
• Bias
• Learning Rate (eta) - Shrinking Learning Rate, Decay Parameters
• Error functions - Entropy, Binary Cross Entropy, Categorical Cross Entropy, KL Divergence, etc.

Artificial Neural Network model used to solve the most complex data where the pattern cannot be defined using explainable models. Neural Networks are used to solve deep learning problems as well. Artificial Neural Network (ANN), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN) are the types of Neural Networks, you will understand the difference among these Networks and their applications in real-time. Learn about Gradient Descent Algorithm and its optimization techniques to reduce the error to better fit the data.

• Artificial Neural networks
• ANN structure
• Gradient Descent Algorithms - Batch GD, SGD, Mini-batch SGD
• Backward propagation
• Network Topology
• Principles of Gradient descent (Manual Calculation)
• Momentum, Nesterov Momentum
• Optimization methods: Adam, Adagrad, Adadelta, RMSProp
• CNN - Convolutional Neural Network
• RNN - Recurrent Neural Network

As part of this module learn about another Deep Learning algorithm SVM which is also a black box technique. SVM is about creating boundaries for classifying data in multidimensional spaces. These boundaries are called hyperplanes which may be linear or non-linear boundaries which segregate the categories to a maximum margin possible. Learn about kernel tricks application to convert the data into high dimensional spaces to classify the non-linear spaces into linearly separable data.

• Support Vector Machines
• Classification Hyperplanes
• Best fit "boundary"
• Kernel Tricks - Linear, RBF, etc.
• Non-Linear Kernel Tricks
• Avoiding overfitting in SVM
• Regularization techniques in SVM

Kaplan Meier method and life tables are used to estimate the time before the event occurs. Survival analysis is about analyzing this duration or time before the event. Real-time applications of survival analysis in customer churn, medical sciences and other sectors is discussed as part of this module. Learn how survival analysis techniques can be used to understand the effect of the features on the event using Kaplan Meier survival plot.

• Examples of Survival Analysis
• Time to event
• Censoring
• Survival, Hazard, Cumulative Hazard Functions
• Introduction to Parametric and non-parametric functions

Time series analysis is performed on the data which is collected with respect to time. The response variable is affected by time. Understand the time series components, Level, Trend, Seasonality, Noise and methods to identify them in a time series data. The different forecasting methods available to handle the estimation of the response variable based on the condition of whether the past is equal to the future or not will be introduced in this module. In this first module of forecasting, you will learn the application of Model-based forecasting techniques.

• Introduction to time series data
• Steps to forecasting
• Components to time series data
• Scatter plot and Time Plot
• Lag Plot
• ACF - Auto-Correlation Function / Correlogram
• Visualization principles
• Naïve forecast methods
• Errors in the forecast and it metrics - ME, MAD, MSE, RMSE, MPE, MAPE
• Model-Based approaches
  • Linear Model
  • Exponential Model
  • Quadratic Model
  • Additive Seasonality
  • Multiplicative Seasonality
• Model-Based approaches Continued
• AR (Auto-Regressive) model for errors
• Random walk

In this continuation module of forecasting learn about data-driven forecasting techniques. Learn about ARMA and ARIMA models which combine model-based and data-driven techniques. Understand the smoothing techniques and variations of these techniques. Get introduced to the concept of de-trending and deseasonalize the data to make it stationary. You will learn about seasonal index calculations which are used for reseasonalize the result obtained by smoothing models.

• ARMA (Auto-Regressive Moving Average), Order p and q
• ARIMA (Auto-Regressive Integrated Moving Average), Order p, d, and q
• A data-driven approach to forecasting
• Smoothing techniques
  • Moving Average
  • Exponential Smoothing
  • Holt's / Double Exponential Smoothing
  • Winters / Holt-Winters
• De-seasoning and de-trending
• Econometric Models
• Forecasting using Python
• Forecasting using R