Workflow Element Store

  1. APIs and Data Feeds
  2. Feedback Data
  3. WebScraping
  4. Surveys and Questionnaires
  5. Mobile Applications or IoT Applications
  6. Data bases - NoSQL
  7. Data Bases - SQL
  8. Experiments (DoE)
  9. Flat files
  10. Public Datasets
  11. Data Collaboration and Partnerships
  1. MS SQL server
  2. Apache Kafka
  3. Azure Streaming Analytics
  4. GCP Data Fusion
  5. Oracle DB
  6. RDBMS
  7. AWS RDS
  8. s3
  9. AWS Kinesis
  10. Azure blob storage
  11. Azure Synapse
  12. MySQL
  13. MongoDB
  14. GCP BigQuery
  15. AWS Redshift
  16. GCP Dataflow
  17. ETL/ELT pipeline
  18. AWS Glue
  19. Azure ADF
  20. PostgreSQL
  21. GCS
  1. Augmentation
  2. Annotation
  3. Dealing with Outliers
  4. Handling Time-Series Data
  5. Dimensionality Reduction
  6. Handling Imbalanced Classes
  7. Handling Missing Data
  8. Feature Selection
  9. Handling Categorical Data
  10. Feature Extraction from Images
  11. Domain-Specific Feature Engineering
  12. Interaction Features
  13. Data Transformations
  14. Binning / Discretization
  15. Data Scaling and Normalization
  16. Time-Based Features
  17. Handling Noisy Data
  18. Auto-Preprocessing libraries
  19. Polynomial Features
  20. AutoEDA libraries
  21. Textual Feature Extraction
  22. Data Partitioning - Train, Validation, & Test
  1. Batch Normalization
  2. Ensemble Techniques
  3. Clustering
  4. Learning Rate Scheduling
  5. Regular Monitoring and Logging
  6. Forecasting Techniques
  7. Regularization
  8. AutoML
  9. Early Stopping
  10. Natural Language Processing
  11. Regression Analysis
  12. Reinforcement Learning
  13. Evaluation Metrics
  14. Weight Initialization
  15. Recommendation Engine
  16. Cross-Validation
  17. Network Analytics/ GeoSpatial Analytics
  18. External Validation
  19. Cross-Validation
  20. Hyperparameter Tuning
  21. Model Interpretability
  22. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  23. Data Augmentation
  24. Regularization Techniques
  25. Transfer Learning
  26. Multiclass Classification Techniques
  27. Blackbox - Neural Network Models
  28. Transfer Learning
  29. Batch Size Selection
  30. Model Comparison
  31. Association Rules
  32. Word Embeddings
  33. Binary Classification Techniques
  34. Performance Visualization
  1. Data Preprocessing pipeline models
  2. Datawarehouse
  3. Databases
  4. code repository
  5. model registry
  1. Streamlit
  2. Alerting and Notification
  3. Containerization
  4. Performance Metrics
  5. Model Serialization
  6. Model Versioning
  7. Prediction Logging
  8. Bias and Fairness Assessment
  9. Cloud Deployment
  10. Model Health Monitoring
  11. Serverless Computing
  12. Model Drift
  13. Data Drift Monitoring
  14. FastAPI
  15. Flask
  16. Concept Drift Detection
  17. Feedback Collection
  18. Edge Deployment
ML Workflow Intermediate - Architecture
  • Element belongs to model
  • Element not belongs to model
Training Pipeline

Data Collection

API Stream

Web crawler

API Stream

Web crawler

Selenium

Data Ingestion

Data Landing Zone

Store Data from all the Sources

Data Cleaning / Preprocessing

Derived & Base features

Data Training & Modelling

Inference Pipeline

Input Data for Forecasting

Input Data

Cleaned & Processed Data

Inference