Workflow Element Store

  1. Augmentation
  2. Data Scaling and Normalization
  3. Domain-Specific Feature Engineering
  4. Textual Feature Extraction
  5. AutoEDA libraries
  6. Data Transformations
  7. Interaction Features
  8. Dimensionality Reduction
  9. Polynomial Features
  10. Dealing with Outliers
  11. Handling Noisy Data
  12. Feature Extraction from Images
  13. Handling Imbalanced Classes
  14. Handling Missing Data
  15. Binning / Discretization
  16. Feature Selection
  17. Time-Based Features
  18. Handling Categorical Data
  19. Data Partitioning - Train, Validation, & Test
  20. Handling Time-Series Data
  21. Auto-Preprocessing libraries
  22. Annotation
  1. Evaluation Metrics
  2. Reinforcement Learning
  3. Early Stopping
  4. Regularization
  5. Transfer Learning
  6. Association Rules
  7. Transfer Learning
  8. Regularization Techniques
  9. Multiclass Classification Techniques
  10. Hyperparameter Tuning
  11. Performance Visualization
  12. Regular Monitoring and Logging
  13. Forecasting Techniques
  14. Recommendation Engine
  15. Blackbox - Neural Network Models
  16. Cross-Validation
  17. Word Embeddings
  18. Weight Initialization
  19. Batch Normalization
  20. AutoML
  21. Cross-Validation
  22. Model Interpretability
  23. Batch Size Selection
  24. Learning Rate Scheduling
  25. External Validation
  26. Binary Classification Techniques
  27. Ensemble Techniques
  28. Data Augmentation
  29. Model Comparison
  30. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  31. Clustering
  32. Natural Language Processing
  33. Regression Analysis
  34. Network Analytics/ GeoSpatial Analytics
  1. Databases
  2. Data Preprocessing pipeline models
  3. model registry
  4. Evidently.ai
  5. Datawarehouse
  6. Apache Airflow
  7. Kafka Brokers
  8. code repository
  9. Github
  10. Github Actions
ML Workflow Advanced - Architecture
  • Element belongs to model
  • Element not belongs to model
Data Sources

Data Sources

Streaming Data

Streaming Data

Batch Data

Batch Data

Cloud Storage

Cloud Storage

Labeled Data

Labeled Data

Feature Engineering Pipeline

Feature Engineering Pipeline

Experimentation

Experimentation

ML Model

ML Model

Repository

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Offline DB Online DB

Orchestration Component

Artifact Store

CI/CD Component

Model Registry

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Automation ML Workflow Pipeline

Monitoring Component

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)