Data Science for Internal Auditors

• Introduction to Python Programming
• Installation of Python & Associated Packages
• Graphical User Interface
• Installation of Anaconda Python
• Setting Up Python Environment
• Data Types
• Operators in Python
  • Arithmetic operators
  • Relational operators
  • Logical operators
  • Assignment operators
  • Bitwise operators
  • Membership operators
  • Identity operators
  • Check out the Top Python Programming Interview Questions and Answers here.
• Data structures
  • Vectors
  • Matrix
  • Arrays
  • Lists
  • Tuple
  • Sets
  • String Representation
  • Arithmetic Operators
  • Boolean Values
  • Dictionary
• Conditional Statements
  • if statement
  • if - else statement
  • if - elif statement
  • Nest if-else
  • Multiple if
  • Switch
• Loops
  • While loop
  • For loop
  • Range()
  • Iterator and generator Introduction
  • For – else
  • Break
• Functions
  • Purpose of a function
  • Defining a function
  • Calling a function
  • Function parameter passing
    • Formal arguments
    • Actual arguments
    • Positional arguments
    • Keyword arguments
    • Variable arguments
    • Variable keyword arguments
    • Use-Case *args, **kwargs
• Function call stack
  • Locals()
  • Globals()
• Stackframe
• Modules
  • Python Code Files
  • Importing functions from another file
  • __name__: Preventing unwanted code execution
  • Importing from a folder
  • Folders Vs Packages
  • __init__.py
  • Namespace
  • __all__
  • Import *
  • Recursive imports
• File Handling
• Exception Handling
• Regular expressions
• Oops concepts
• Classes and Objects
• Inheritance and Polymorphism
• Multi-Threading

• What is a Database
• Types of Databases
• DBMS vs RDBMS
• DBMS Architecture
• Normalisation & Denormalization
• Install PostgreSQL
• Install MySQL
• Data Models
• DBMS Language
• ACID Properties in DBMS
• What is SQL
• SQL Data Types
• SQL commands
• SQL Operators
• SQL Keys
• SQL Joins
• GROUP BY, HAVING, ORDER BY
• Subqueries with select, insert, update, delete
• atements?
• Views in SQL
• SQL Set Operations and Types
• SQL functions
• SQL Triggers
• Introduction to NoSQL Concepts
• SQL vs NoSQL
• Database connection SQL to Python
• Check out the SQL for Data Science One Step Solution for Beginners here.

Learn about insights on how data is assisting organizations to make informed data-driven decisions. Gathering the details about the problem statement would be the first step of the project. Learn the know-how of the Business understanding stage. Deep dive into the finer aspects of the management methodology to learn about objectives, constraints, success criteria, and the project charter. The essential task of understanding business Data and its characteristics is to help you plan for the upcoming stages of development. Check out the CRISP - Business Understanding here.

• All About 360DigiTMG & Innodatatics Inc., USA
• Dos and Don'ts as a participant
• Introduction to Big Data Analytics
• Data and its uses – a case study (Grocery store)
• Interactive marketing using data & IoT – A case study
• Course outline, road map, and takeaways from the course
• Stages of Analytics - Descriptive, Predictive, Prescriptive, etc.
• Cross-Industry Standard Process for Data Mining

• Typecasting
• Handling Duplicates
• Outlier Analysis/Treatment
• Zero or Near Zero Variance Features
• Missing Values
• Discretization / Binning / Grouping
• Encoding: Dummy Variable Creation
• Transformation
• Scaling: Standardization / Normalization

In this module, you will learn about dealing with the Data after the Collection. Learn to extract meaningful information about Data by performing Uni-variate analysis which is the preliminary step to churn the data. The task is also called Descriptive Analytics or also known as exploratory data analysis. In this module, you also are introduced to statistical calculations which are used to derive information along with Visualizations to show the information in graphs/plots

• Machine Learning project management methodology
• Data Collection - Surveys and Design of Experiments
• Data Types namely Continuous, Discrete, Categorical, Count, Qualitative, Quantitative and its identification and application
• Further classification of data in terms of Nominal, Ordinal, Interval & Ratio types
• Balanced versus Imbalanced datasets
• Cross Sectional versus Time Series vs Panel / Longitudinal Data
• Batch Processing vs Real Time Processing
• Structured versus Unstructured vs Semi-Structured Data
• Big vs Not-Big Data
• Data Cleaning / Preparation - Outlier Analysis, Missing Values Imputation Techniques, Transformations, Normalization / Standardization, Discretization
• Sampling techniques for handling Balanced vs. Imbalanced Datasets
• What is the Sampling Funnel and its application and its components?
  • Population
  • Sampling frame
  • Simple random sampling
  • Sample
• Measures of Central Tendency & Dispersion
  • Population
  • Mean/Average, Median, Mode
  • Variance, Standard Deviation, Range

The raw Data collected from different sources may have different formats, values, shapes, or characteristics. Cleansing, or Data Preparation, Data Munging, Data Wrapping, etc., are the next steps in the Data handling stage. The objective of this stage is to transform the Data into an easily consumable format for the next stages of development.

• Feature Engineering on Numeric / Non-numeric Data
• Feature Extraction
• Feature Selection

• What is Power BI?
  • Introduction to Power BI
  • Overview of Power BI
  • Architecture of PowerBI
  • PowerBI and Plans
  • Installation and introduction to PowerBI
• Transforming Data using Power BI Desktop
  • Importing data
  • Changing Database
  • Data Types in PowerBI
  • Basic Transformations
  • Managing Query Groups
  • Splitting Columns
  • Changing Data Types
  • Working with Dates
  • Removing and Reordering Columns
  • Conditional Columns
  • Custom columns
  • Connecting to Files in a Folder
  • Merge Queries
  • Query Dependency View
  • Transforming Less Structured Data
  • Query Parameters
  • Column profiling
  • Query Performance Analytics
  • M-Language

Learn the preliminaries of the Mathematical / Statistical concepts which are the foundation of techniques used for churning the Data. You will revise the primary academic concepts of foundational mathematics and Linear Algebra basics. In this module, you will understand the importance of Data Optimization concepts in Machine Learning development. Check out the Mathematical Foundations here.

• Data Optimization
• Derivatives
• Linear Algebra
• Matrix Operations

Data mining unsupervised techniques are used as EDA techniques to derive insights from the business data. In this first module of unsupervised learning, get introduced to clustering algorithms. Learn about different approaches for data segregation to create homogeneous groups of data. In hierarchical clustering, K means clustering is the most used clustering algorithm. Understand the different mathematical approaches to perform data segregation. Also, learn about variations in K-means clustering like K-medoids, and K-mode techniques, and learn to handle large data sets using the CLARA technique.

• Clustering 101
• Distance Metrics
• Hierarchical Clustering
• Non-Hierarchical Clustering
• DBSCAN
• Clustering Evaluation metrics

Dimension Reduction (PCA and SVD) / Factor Analysis Description: Learn to handle high dimensional data. The performance will be hit when the data has a high number of dimensions and machine learning techniques training becomes very complex, as part of this module you will learn to apply data reduction techniques without any variable deletion. Learn the advantages of dimensional reduction techniques. Also, learn about yet another technique called Factor Analysis.

• Prinicipal Component Analysis (PCA)
• Singular Value Decomposition (SVD)

Learn to measure the relationship between entities. Bundle offers are defined based on this measure of dependency between products. Understand the metrics Support, Confidence, and Lift used to define the rules with the help of the Apriori algorithm. Learn the pros and cons of each of the metrics used in Association rules.

• Association rules mining 101
• Measurement Metrics
• Support
• Confidence
• Lift

• User Based Collaborative Filtering
• Similarity Metrics
• Item Based Collaborative Filtering
• Search Based Methods
• SVD Method

The study of a network with quantifiable values is known as network analytics. The vertex and edge are the nodes and connection of a network, learn about the statistics used to calculate the value of each node in the network. You will also learn about the google page ranking algorithm as part of this module.

• Entities of a Network
• Properties of the Components of a Network
• Measure the value of a Network
• Community Detection Algorithms

Learn to analyse unstructured textual data to derive meaningful insights. Understand the language quirks to perform data cleansing, extract features using a bag of words and construct the key-value pair matrix called DTM. Learn to understand the sentiment of customers from their feedback to take appropriate actions. Advanced concepts of text mining will also be discussed which help to interpret the context of the raw text data. Topic models using LDA algorithm, emotion mining using lexicons are discussed as part of NLP module.

• Sources of data
• Bag of words
• Pre-processing, corpus Document Term Matrix (DTM) & TDM
• Word Clouds
• Corpus-level word clouds
• Sentiment Analysis
• Positive Word clouds
• Negative word clouds
• Unigram, Bigram, Trigram
• Semantic network
• Extract, user reviews of the product/services from Amazon and tweets from Twitter
• Install Libraries from Shell
• Extraction and text analytics in Python
• LDA / Latent Dirichlet Allocation
• Topic Modelling
• Sentiment Extraction
• Lexicons & Emotion Mining
• Check out the Text Mining Interview Questions and Answers here.

• Machine Learning primer
• Difference between Regression and Classification
• Evaluation Strategies
• Hyper Parameters
• Metrics
• Overfitting and Underfitting

Revise Bayes theorem to develop a classification technique for Machine learning. In this tutorial, you will learn about joint probability and its applications. Learn how to predict whether an incoming email is spam or a ham email. Learn about Bayesian probability and its applications in solving complex business problems.

• Probability – Recap
• Bayes Rule
• Naïve Bayes Classifier
• Text Classification using Naive Bayes
• Checking for Underfitting and Overfitting in Naive Bayes
• Generalization and Regulation Techniques to avoid overfitting in Naive Bayes
• Check out the Naive Bayes Algorithm here.

k Nearest Neighbor algorithm is a distance-based machine learning algorithm. Learn to classify the dependent variable using the appropriate k value. The KNN Classifier also known as a lazy learner is a very popular algorithm and one of the easiest for application.

• Deciding the K value
• Thumb rule in choosing the K value.
• Building a KNN model by splitting the data
• Checking for Underfitting and Overfitting in KNN
• Generalization and Regulation Techniques to avoid overfitting in KNN

In this tutorial, you will learn in detail about the continuous probability distribution. Understand the properties of a continuous random variable and its distribution under normal conditions. To identify the properties of a continuous random variable, statisticians have defined a variable as a standard, learning the properties of the standard variable and its distribution. You will learn to check if a continuous random variable is following normal distribution using a normal Q-Q plot. Learn the science behind the estimation of value for a population using sample data.

• Probability & Probability Distribution
• Continuous Probability Distribution / Probability Density Function
• Discrete Probability Distribution / Probability Mass Function
• Normal Distribution
• Standard Normal Distribution / Z distribution
• Z scores and the Z table
• QQ Plot / Quantile - Quantile plot
• Sampling Variation
• Central Limit Theorem
• Sample size calculator
• Confidence interval - concept
• Confidence interval with sigma
• T-distribution Table / Student's-t distribution / T table
• Confidence interval
• Population parameter with Standard deviation known
• Population parameter with Standard deviation not known

Learn to frame business statements by making assumptions. Understand how to perform testing of these assumptions to make decisions for business problems. Learn about different types of Hypothesis testing and its statistics. You will learn the different conditions of the Hypothesis table, namely Null Hypothesis, Alternative hypothesis, Type I error, and Type II error. The prerequisites for conducting a Hypothesis test, and interpretation of the results will be discussed in this module.

• Formulating a Hypothesis
• Choosing Null and Alternative Hypotheses
• Type I or Alpha Error and Type II or Beta Error
• Confidence Level, Significance Level, Power of Test
• Comparative study of sample proportions using Hypothesis testing
• 2 Sample t-test
• ANOVA
• 2 Proportion test
• Chi-Square test

Data Mining supervised learning is all about making predictions for an unknown dependent variable using mathematical equations explaining the relationship with independent variables. Revisit the school math with the equation of a straight line. Learn about the components of Linear Regression with the equation of the regression line. Get introduced to Linear Regression analysis with a use case for the prediction of a continuous dependent variable. Understand about ordinary least squares technique.

• Scatter diagram
• Correlation analysis
• Correlation coefficient
• Ordinary least squares
• Principles of regression
• Simple Linear Regression
• Exponential Regression, Logarithmic Regression, Quadratic or Polynomial Regression
• Confidence Interval versus Prediction Interval
• Heteroscedasticity / Equal Variance
• Check out the Linear Regression Interview Questions and Answers here.

In the continuation of the Regression analysis study, you will learn how to deal with multiple independent variables affecting the dependent variable. Learn about the conditions and assumptions to perform linear regression analysis and the workarounds used to follow the conditions. Understand the steps required to perform the evaluation of the model and to improvise the prediction accuracies. You will be introduced to concepts of variance and bias.

• LINE assumption
• Linearity
• Independence
• Normality
• Equal Variance / Homoscedasticity
• Collinearity (Variance Inflation Factor)
• Multiple Linear Regression
• Model Quality metrics
• Deletion Diagnostics
• Check out the Linear Regression Interview Questions here.

You have learned about predicting a continuous dependent variable. As part of this module, you will continue to learn Regression techniques applied to predict attribute Data. Learn about the principles of the logistic regression model, understand the sigmoid curve, and the usage of cut-off value to interpret the probable outcome of the logistic regression model. Learn about the confusion matrix and its parameters to evaluate the outcome of the prediction model. Also, learn about maximum likelihood estimation.

• Principles of Logistic regression
• Types of Logistic regression
• Assumption & Steps in Logistic regression
• Analysis of Simple logistic regression results
• Multiple Logistic regression
• Confusion matrix
• False Positive, False Negative
• True Positive, True Negative
• Sensitivity, Recall, Specificity, F1
• Receiver operating characteristics curve (ROC curve)
• Precision Recall (P-R) curve
• Lift charts and Gain charts
• Check out the Logistic Regression Interview Questions and Answers here.

Learn about overfitting and underfitting conditions for prediction models developed. We need to strike the right balance between overfitting and underfitting, learn about regularization techniques L1 norm and L2 norm used to reduce these abnormal conditions. The regression techniques of Lasso and Ridge techniques are discussed in this module.

• Understanding Overfitting (Variance) vs. Underfitting (Bias)
• Generalization error and Regularization techniques
• Different Error functions, Loss functions, or Cost functions
• Lasso Regression
• Ridge Regression
• Check out the Lasso and Ridge Regression Interview Questions and Answers here.

Extension to logistic regression We have multinomial and Ordinal Logistic regression techniques used to predict multiple categorical outcomes. Understand the concept of multi-logit equations, baseline, and making classifications using probability outcomes. Learn about handling multiple categories in output variables including nominal as well as ordinal data.

• Logit and Log-Likelihood
• Category Baselining
• Modeling Nominal categorical data
• Handling Ordinal Categorical Data
• Interpreting the results of coefficient values

As part of this module, you learn further different regression techniques used for predicting discrete data. These regression techniques are used to analyze the numeric data known as count data. Based on the discrete probability distributions namely Poisson, negative binomial distribution the regression models try to fit the data to these distributions. Alternatively, when excessive zeros exist in the dependent variable, zero-inflated models are preferred, you will learn the types of zero-inflated models used to fit excessive zeros data.

• Poisson Regression
• Poisson Regression with Offset
• Negative Binomial Regression
• Treatment of data with Excessive Zeros
• Zero-inflated Poisson
• Zero-inflated Negative Binomial
• Hurdle Model

Support Vector Machines / Large-Margin / Max-Margin Classifier

• Hyperplanes
• Best Fit "boundary"
• Linear Support Vector Machine using Maximum Margin
• SVM for Noisy Data
• Non- Linear Space Classification
• Non-Linear Kernel Tricks
• Linear Kernel
• Polynomial
• Sigmoid
• Gaussian RBF
• SVM for Multi-Class Classification
• One vs. All
• One vs. One
• Directed Acyclic Graph (DAG) SVM

Kaplan Meier method and life tables are used to estimate the time before the event occurs. Survival analysis is about analyzing the duration of time before the event. Real-time applications of survival analysis in customer churn, medical sciences, and other sectors are discussed as part of this module. Learn how survival analysis techniques can be used to understand the effect of the features on the event using the Kaplan-Meier survival plot.

• Examples of Survival Analysis
• Time to event
• Censoring
• Survival, Hazard, and Cumulative Hazard Functions
• Introduction to Parametric and non-parametric functions

Decision Tree models are some of the most powerful classifier algorithms based on classification rules. In this tutorial, you will learn about deriving the rules for classifying the dependent variable by constructing the best tree using statistical measures to capture the information from each of the attributes.

• Elements of classification tree - Root node, Child Node, Leaf Node, etc.
• Greedy algorithm
• Measure of Entropy
• Attribute selection using Information gain
• Decision Tree C5.0 and understanding various arguments
• Checking for Underfitting and Overfitting in Decision Tree
• Pruning – Pre and Post Prune techniques
• Generalization and Regulation Techniques to avoid overfitting in Decision Tree
• Random Forest and understanding various arguments
• Checking for Underfitting and Overfitting in Random Forest
• Generalization and Regulation Techniques to avoid overfitting in Random Forest
• Check out the Decision Tree Questions here.

Learn about improving the reliability and accuracy of decision tree models using ensemble techniques. Bagging and Boosting are the go-to techniques in ensemble techniques. The parallel and sequential approaches taken in Bagging and Boosting methods are discussed in this module. Random forest is yet another ensemble technique constructed using multiple Decision trees and the outcome is drawn from the aggregating the results obtained from these combinations of trees. The Boosting algorithms AdaBoost and Extreme Gradient Boosting are discussed as part of this continuation module. You will also learn about stacking methods. Learn about these algorithms which are providing unprecedented accuracy and helping many aspiring data scientists win first place in various competitions such as Kaggle, CrowdAnalytix, etc.

• Overfitting
• Underfitting
• Voting
• Stacking
• Bagging
• Random Forest
• Boosting
• AdaBoost / Adaptive Boosting Algorithm
• Checking for Underfitting and Overfitting in AdaBoost
• Generalization and Regulation Techniques to avoid overfitting in AdaBoost
• Gradient Boosting Algorithm
• Checking for Underfitting and Overfitting in Gradient Boosting
• Generalization and Regulation Techniques to avoid overfitting in Gradient Boosting
• Extreme Gradient Boosting (XGB) Algorithm
• Checking for Underfitting and Overfitting in XGB
• Generalization and Regulation Techniques to avoid overfitting in XGB
• Check out the Ensemble Techniques Interview Questions here.

Time series analysis is performed on the data which is collected with respect to time. The response variable is affected by time. Understand the time series components, Level, Trend, Seasonality, Noise, and methods to identify them in a time series data. The different forecasting methods available to handle the estimation of the response variable based on the condition of whether the past is equal to the future or not will be introduced in this module. In this first module of forecasting, you will learn the application of Model-based forecasting techniques.

• Introduction to time series data
• Steps to forecasting
• Components to time series data
• Scatter plot and Time Plot
• Lag Plot
• ACF - Auto-Correlation Function / Correlogram
• Visualization principles
• Naïve forecast methods
• Errors in the forecast and it metrics - ME, MAD, MSE, RMSE, MPE, MAPE
• Model-Based approaches
• Linear Model
• Exponential Model
• Quadratic Model
• Additive Seasonality
• Multiplicative Seasonality
• Model-Based approaches Continued
• AR (Auto-Regressive) model for errors
• Random walk
• Check out the Time Series Interview Questions here.

In this continuation module of forecasting learn about data-driven forecasting techniques. Learn about ARMA and ARIMA models which combine model-based and data-driven techniques. Understand the smoothing techniques and variations of these techniques. Get introduced to the concept of de-trending and de-seasonalize the data to make it stationary. You will learn about seasonal index calculations which are used to re-seasonalize the result obtained by smoothing models.

• ARMA (Auto-Regressive Moving Average), Order p and q
• ARIMA (Auto-Regressive Integrated Moving Average), Order p, d, and q
• A data-driven approach to forecasting
• Smoothing techniques
• Moving Average
• Exponential Smoothing
• Holt's / Double Exponential Smoothing
• Winters / Holt-Winters
• De-seasoning and de-trending
• Seasonal Indexes

The Perceptron Algorithm is defined based on a biological brain model. You will talk about the parameters used in the perceptron algorithm which is the foundation of developing much complex neural network models for AI applications. Understand the application of perceptron algorithms to classify binary data in a linearly separable scenario.

• Neurons of a Biological Brain
• Artificial Neuron
• Perceptron
• Perceptron Algorithm
• Use case to classify a linearly separable data
• Multilayer Perceptron to handle non-linear data

Neural Network is a black box technique used for deep learning models. Learn the logic of training and weights calculations using various parameters and their tuning. Understand the activation function and integration functions used in developing a Artificial Neural Network.

• Integration functions
• Activation functions
• Weights
• Bias
• Learning Rate (eta) - Shrinking Learning Rate, Decay Parameters
• Error functions - Entropy, Binary Cross Entropy, Categorical Cross Entropy, KL Divergence, etc.

• Artificial Neural Networks
• ANN Structure
• Error Surface
• Gradient Descent Algorithm
• Backward Propagation
• Network Topology
• Principles of Gradient Descent (Manual Calculation)
• Learning Rate (eta)
• Batch Gradient Descent
• Stochastic Gradient Descent
• Minibatch Stochastic Gradient Descent
• Optimization Methods: Adagrad, Adadelta, RMSprop, Adam
• Convolution Neural Network (CNN)
• ImageNet Challenge – Winning Architectures
• Parameter Explosion with MLPs
• Convolution Networks
• Recurrent Neural Network
• Language Models
• Traditional Language Model
• Disadvantages of MLP
• Back Propagation Through Time
• Long Short-Term Memory (LSTM)
• Gated Recurrent Network (GRU)