Workflow Element Store

  1. Textual Feature Extraction
  2. Feature Extraction from Images
  3. Handling Missing Data
  4. Handling Categorical Data
  5. Dimensionality Reduction
  6. AutoEDA libraries
  7. Binning / Discretization
  8. Handling Imbalanced Classes
  9. Dealing with Outliers
  10. Feature Selection
  11. Domain-Specific Feature Engineering
  12. Data Transformations
  13. Annotation
  14. Handling Noisy Data
  15. Handling Time-Series Data
  16. Time-Based Features
  17. Augmentation
  18. Data Partitioning - Train, Validation, & Test
  19. Polynomial Features
  20. Auto-Preprocessing libraries
  21. Data Scaling and Normalization
  22. Interaction Features
  1. Batch Size Selection
  2. Multiclass Classification Techniques
  3. Model Comparison
  4. Hyperparameter Tuning
  5. Clustering
  6. Binary Classification Techniques
  7. Regular Monitoring and Logging
  8. Natural Language Processing
  9. Regression Analysis
  10. Reinforcement Learning
  11. Batch Normalization
  12. Data Augmentation
  13. Transfer Learning
  14. Cross-Validation
  15. Cross-Validation
  16. AutoML
  17. Recommendation Engine
  18. Regularization
  19. Blackbox - Neural Network Models
  20. Evaluation Metrics
  21. Regularization Techniques
  22. Weight Initialization
  23. Network Analytics/ GeoSpatial Analytics
  24. Model Interpretability
  25. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  26. Ensemble Techniques
  27. Word Embeddings
  28. Association Rules
  29. Learning Rate Scheduling
  30. Early Stopping
  31. Forecasting Techniques
  32. External Validation
  33. Transfer Learning
  34. Performance Visualization
  1. Databases
  2. Github Actions
  3. Github
  4. Kafka Brokers
  5. Data Preprocessing pipeline models
  6. model registry
  7. Datawarehouse
  8. Apache Airflow
  9. Evidently.ai
  10. code repository
ML Workflow Advanced - Architecture
  • Element belongs to model
  • Element not belongs to model
Data Sources

Data Sources

Streaming Data

Streaming Data

Batch Data

Batch Data

Cloud Storage

Cloud Storage

Labeled Data

Labeled Data

Feature Engineering Pipeline

Feature Engineering Pipeline

Experimentation

Experimentation

ML Model

ML Model

Repository

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Offline DB Online DB

Orchestration Component

Artifact Store

CI/CD Component

Model Registry

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Automation ML Workflow Pipeline

Monitoring Component

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)