Workflow Element Store

  1. Feature Extraction from Images
  2. Time-Based Features
  3. Binning / Discretization
  4. Auto-Preprocessing libraries
  5. Dimensionality Reduction
  6. Data Partitioning - Train, Validation, & Test
  7. Annotation
  8. Dealing with Outliers
  9. Handling Noisy Data
  10. Handling Time-Series Data
  11. Data Scaling and Normalization
  12. Feature Selection
  13. Handling Categorical Data
  14. Polynomial Features
  15. Handling Imbalanced Classes
  16. AutoEDA libraries
  17. Interaction Features
  18. Augmentation
  19. Textual Feature Extraction
  20. Handling Missing Data
  21. Data Transformations
  22. Domain-Specific Feature Engineering
  1. Learning Rate Scheduling
  2. Regularization Techniques
  3. Batch Size Selection
  4. Early Stopping
  5. Cross-Validation
  6. Clustering
  7. Weight Initialization
  8. Regression Analysis
  9. Hyperparameter Tuning
  10. Binary Classification Techniques
  11. Performance Visualization
  12. Model Comparison
  13. Forecasting Techniques
  14. Transfer Learning
  15. Ensemble Techniques
  16. External Validation
  17. Batch Normalization
  18. AutoML
  19. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  20. Network Analytics/ GeoSpatial Analytics
  21. Regular Monitoring and Logging
  22. Model Interpretability
  23. Data Augmentation
  24. Multiclass Classification Techniques
  25. Association Rules
  26. Cross-Validation
  27. Evaluation Metrics
  28. Transfer Learning
  29. Natural Language Processing
  30. Blackbox - Neural Network Models
  31. Regularization
  32. Reinforcement Learning
  33. Recommendation Engine
  34. Word Embeddings
  1. Datawarehouse
  2. Kafka Brokers
  3. model registry
  4. Evidently.ai
  5. Data Preprocessing pipeline models
  6. Databases
  7. code repository
  8. Github Actions
  9. Apache Airflow
  10. Github
ML Workflow - Architecture
  • Element belongs to model
  • Element not belongs to model

Data Sources

Streaming Data

Batch Data

Cloud Storage

Labeled Data

Feature Engineering Pipeline

Experimentation

ML Model

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Orchestration Component

Artifact Store

CI/CD Component

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)