Workflow Element Store

  1. Data Scaling and Normalization
  2. Handling Missing Data
  3. AutoEDA libraries
  4. Polynomial Features
  5. Binning / Discretization
  6. Domain-Specific Feature Engineering
  7. Annotation
  8. Textual Feature Extraction
  9. Auto-Preprocessing libraries
  10. Interaction Features
  11. Data Partitioning - Train, Validation, & Test
  12. Handling Time-Series Data
  13. Handling Categorical Data
  14. Time-Based Features
  15. Feature Selection
  16. Dealing with Outliers
  17. Augmentation
  18. Handling Noisy Data
  19. Handling Imbalanced Classes
  20. Dimensionality Reduction
  21. Feature Extraction from Images
  22. Data Transformations
  1. Learning Rate Scheduling
  2. Reinforcement Learning
  3. Blackbox - Neural Network Models
  4. Cross-Validation
  5. Hyperparameter Tuning
  6. Data Augmentation
  7. Model Interpretability
  8. Cross-Validation
  9. Ensemble Techniques
  10. Transfer Learning
  11. Batch Normalization
  12. Multiclass Classification Techniques
  13. Association Rules
  14. Clustering
  15. Natural Language Processing
  16. Model Comparison
  17. Forecasting Techniques
  18. AutoML
  19. Evaluation Metrics
  20. Word Embeddings
  21. Early Stopping
  22. Recommendation Engine
  23. Performance Visualization
  24. GridSearchCV, RandomisedSearchCV, BayesianSearchCV
  25. Batch Size Selection
  26. Regularization
  27. Regularization Techniques
  28. Binary Classification Techniques
  29. Transfer Learning
  30. External Validation
  31. Regression Analysis
  32. Weight Initialization
  33. Network Analytics/ GeoSpatial Analytics
  34. Regular Monitoring and Logging
  1. Datawarehouse
  2. Github
  3. Databases
  4. Github Actions
  5. Apache Airflow
  6. code repository
  7. Evidently.ai
  8. model registry
  9. Data Preprocessing pipeline models
  10. Kafka Brokers
ML Workflow - Architecture
  • Element belongs to model
  • Element not belongs to model
Validate Export

Data Sources

Streaming Data

Batch Data

Cloud Storage

Labeled Data

Feature Engineering Pipeline

Experimentation

ML Model

Repository

CI/CD component

Continuous integration/Continuous delivery

Continuous deployment

Artifact Store

Feature Store System

Orchestration Component

Artifact Store

CI/CD Component

Scheduler

Workflow orchestration component

Automation ML Workflow Pipeline

Monitoring Component

Model serving component

(Prediction on new batch or streaming data)