Workflow Element Store

  1. Data Generation
  2. Surveys and Questionnaires
  3. Crowdsourcing
  4. APIs and Data Feeds
  5. Structured Data (Tabular)
  6. WebScraping
  7. Data Pre-existing
  8. Unstructured data (Images / Videos)
  9. Data Logging
  10. Unstructured data (Audio)
  11. Mobile Applications or IoT Applications
  12. Data Collaboration and Partnerships
  13. Public Datasets
  1. GCP BigQuery
  2. NoSQL DB
  3. Azure Data Warehouse
  4. GCS
  5. Oracle DB
  6. MS SQL server
  7. AWS Redshift
  8. MySQL
  9. S3
  10. PostgreSQL
  11. RDBMS
  12. Informatica
  13. Azure blob storage
  1. Data Scaling and Normalization
  2. Feature Selection
  3. Handling Categorical Data
  4. Polynomial Features
  5. Handling Missing Data
  6. Auto-Preprocessing libraries
  7. Handling Time-Series Data
  8. Handling Noisy Data
  9. Dealing with Outliers
  10. Encoding Categorical Variables
  11. Time-Based Features
  12. Feature Extraction from Images
  13. Dimensionality Reduction
  14. Textual Feature Extraction
  15. Binning
  16. Dimensionality Reduction
  17. Domain-Specific Feature Engineering
  18. AutoEDA libraries
  19. Interaction Features
  20. Handling Imbalanced Classes
  21. Data Scaling and Normalization
  22. Logarithmic Transform
  1. Time Series Anaysis
  2. Data Partitioning
  3. Supervised Learning-binary classification
  4. Blackbox Techniques
  5. Ensemble Techniques
  6. Train-Test Split
  7. Unsupervised Learning
  8. Supervised Learning-multiclass classification
  9. Supervised Learning-Regression
  10. Forecasting
  1. Learning Rate Scheduling
  2. Batch Size Selection
  3. Regularization
  4. Batch Normalization
  5. Train-Test Split
  6. Regular Monitoring and Logging
  7. Weight Initialization
  8. Early Stopping
  9. Ensemble Methods
  10. Gradient Clipping
  11. Data Partition-sequential
  12. Hyperparameter Tuning
  13. Data Augmentation
  14. Cross-Validation
  15. Transfer Learning
  1. Hyperparameter Tuning
  2. Data Partitioning
  3. Model Interpretability
  4. Cross-Validation
  5. External Validation
  6. Model Comparison
  7. Train-Test Split
  8. Evaluation Metrics
  9. Performance Visualization
  10. Regularization Techniques
  1. Model Health Monitoring
  2. Bias and Fairness Assessment
  3. Documentation and Reporting
  4. Model Versioning
  5. Continuous Integration and Deployment (CI/CD)
  6. Model Drift
  7. Cloud Deployment
  8. Streamlit
  9. Model Monitoring and Maintenance
  10. Serverless Computing
  11. Model Serialization
  12. Concept Drift Detection
  13. Model Retraining and Updating
  14. Performance Metrics
  15. Edge Deployment
  16. Error Analysis
  17. Containerization
  18. Security Considerations
  19. Monitoring and Logging
  20. Web APIs - Flask, FastAPI, etc.
  21. A/B Testing
  22. Prediction Logging
  23. Model Registry
  24. Feedback Collection
  25. Documentation and API Documentation
  26. Data Drift Monitoring
  27. Alerting and Notification
  1. End User Machine
  2. Mobile
ML Workflow Beginner - Architecture
  • Element belongs to model
  • Element not belongs to model

Feature Store
(Online / Offline)

Data Sources

Data Warehouse/ Data Lake

EDA, Data Pre Processing & Feature Engineering

Model Selection

Model Training & Hyper Parameter Tuning

Model Evaluation

Model Deployment

End User Device

Model Registry