Workflow Element Store

  1. Data Scaling and Normalization
  2. Interaction Features
  3. Feature Extraction from Images
  4. Dealing with Outliers
  5. Auto-Preprocessing libraries
  6. Handling Time-Series Data
  7. Dimensionality Reduction
  8. Handling Imbalanced Classes
  9. Binning / Discretization
  10. Data Transformations
  11. Data Partitioning - Train, Validation, & Test
  12. Annotation
  13. Textual Feature Extraction
  14. Feature Selection
  15. Handling Noisy Data
  16. AutoEDA libraries
  17. Augmentation
  18. Handling Categorical Data
  19. Time-Based Features
  20. Handling Missing Data
  21. Polynomial Features
  22. Domain-Specific Feature Engineering
  1. Reinforcement Learning
  2. Recommendation Engine
  3. Data Augmentation
  4. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  5. Blackbox - Neural Network Models
  6. Cross-Validation
  7. External Validation
  8. AutoML
  9. Regularization
  10. Early Stopping
  11. Ensemble Techniques
  12. Learning Rate Scheduling
  13. Regularization Techniques
  14. Model Interpretability
  15. Batch Size Selection
  16. Transfer Learning
  17. Weight Initialization
  18. Evaluation Metrics
  19. Hyperparameter Tuning
  20. Word Embeddings
  21. Batch Normalization
  22. Forecasting Techniques
  23. Natural Language Processing
  24. Association Rules
  25. Multiclass Classification Techniques
  26. Network Analytics/ GeoSpatial Analytics
  27. Regular Monitoring and Logging
  28. Regression Analysis
  29. Binary Classification Techniques
  30. Performance Visualization
  31. Model Comparison
  32. Clustering
  33. Transfer Learning
  34. Cross-Validation
  1. Github
  2. Github Actions
  3. Kafka Brokers
  4. code repository
  5. Databases
  6. model registry
  7. Datawarehouse
  8. Evidently.ai
  9. Apache Airflow
  10. Data Preprocessing pipeline models
ML Workflow Advanced - Architecture
  • Element belongs to model
  • Element not belongs to model
Data Sources

Data Sources

Streaming Data

Streaming Data

Batch Data

Batch Data

Cloud Storage

Cloud Storage

Labeled Data

Labeled Data

Feature Engineering Pipeline

Feature Engineering Pipeline

Experimentation

Experimentation

ML Model

ML Model

Repository

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Offline DB Online DB

Orchestration Component

Artifact Store

CI/CD Component

Model Registry

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Automation ML Workflow Pipeline

Monitoring Component

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)