Workflow Element Store

  1. Data Transformations
  2. Domain-Specific Feature Engineering
  3. Handling Noisy Data
  4. Interaction Features
  5. Feature Extraction from Images
  6. Binning / Discretization
  7. Augmentation
  8. AutoEDA libraries
  9. Handling Time-Series Data
  10. Dealing with Outliers
  11. Dimensionality Reduction
  12. Handling Categorical Data
  13. Annotation
  14. Data Partitioning - Train, Validation, & Test
  15. Auto-Preprocessing libraries
  16. Time-Based Features
  17. Textual Feature Extraction
  18. Data Scaling and Normalization
  19. Feature Selection
  20. Polynomial Features
  21. Handling Missing Data
  22. Handling Imbalanced Classes
  1. Batch Size Selection
  2. Forecasting Techniques
  3. Performance Visualization
  4. Recommendation Engine
  5. External Validation
  6. Evaluation Metrics
  7. Regression Analysis
  8. Cross-Validation
  9. Association Rules
  10. Natural Language Processing
  11. Blackbox - Neural Network Models
  12. Reinforcement Learning
  13. Binary Classification Techniques
  14. Clustering
  15. Learning Rate Scheduling
  16. Hyperparameter Tuning
  17. Regular Monitoring and Logging
  18. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  19. Regularization Techniques
  20. AutoML
  21. Batch Normalization
  22. Data Augmentation
  23. Weight Initialization
  24. Transfer Learning
  25. Model Interpretability
  26. Regularization
  27. Ensemble Techniques
  28. Transfer Learning
  29. Cross-Validation
  30. Model Comparison
  31. Word Embeddings
  32. Network Analytics/ GeoSpatial Analytics
  33. Multiclass Classification Techniques
  34. Early Stopping
  1. Data Preprocessing pipeline models
  2. Github
  3. Apache Airflow
  4. Databases
  5. code repository
  6. Datawarehouse
  7. Evidently.ai
  8. Github Actions
  9. Kafka Brokers
  10. model registry
ML Workflow Advanced - Architecture
  • Element belongs to model
  • Element not belongs to model
Data Sources

Data Sources

Streaming Data

Streaming Data

Batch Data

Batch Data

Cloud Storage

Cloud Storage

Labeled Data

Labeled Data

Feature Engineering Pipeline

Feature Engineering Pipeline

Experimentation

Experimentation

ML Model

ML Model

Repository

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Orchestration Component

Artifact Store

CI/CD Component

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Automation ML Workflow Pipeline

Monitoring Component

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)