Workflow Element Store

  1. Public Datasets
  2. Data Pre-existing
  3. Data Logging
  4. Unstructured data (Audio)
  5. Structured Data (Tabular)
  6. APIs and Data Feeds
  7. Data Generation
  8. WebScraping
  9. Data Collaboration and Partnerships
  10. Unstructured data (Images / Videos)
  11. Surveys and Questionnaires
  12. Crowdsourcing
  13. Mobile Applications or IoT Applications
  1. MySQL
  2. GCP BigQuery
  3. S3
  4. Azure Data Warehouse
  5. NoSQL DB
  6. Azure blob storage
  7. Oracle DB
  8. RDBMS
  9. GCS
  10. MS SQL server
  11. PostgreSQL
  12. AWS Redshift
  13. Informatica
  1. Handling Time-Series Data
  2. Dimensionality Reduction
  3. Handling Missing Data
  4. Textual Feature Extraction
  5. Time-Based Features
  6. Data Scaling and Normalization
  7. Dimensionality Reduction
  8. Handling Categorical Data
  9. Auto-Preprocessing libraries
  10. Dealing with Outliers
  11. Binning
  12. Interaction Features
  13. Logarithmic Transform
  14. Feature Extraction from Images
  15. AutoEDA libraries
  16. Domain-Specific Feature Engineering
  17. Handling Imbalanced Classes
  18. Encoding Categorical Variables
  19. Handling Noisy Data
  20. Data Scaling and Normalization
  21. Polynomial Features
  22. Feature Selection
  1. Time Series Anaysis
  2. Supervised Learning-multiclass classification
  3. Data Partitioning
  4. Train-Test Split
  5. Supervised Learning-binary classification
  6. Ensemble Techniques
  7. Supervised Learning-Regression
  8. Blackbox Techniques
  9. Unsupervised Learning
  10. Forecasting
  1. Regular Monitoring and Logging
  2. Weight Initialization
  3. Learning Rate Scheduling
  4. Train-Test Split
  5. Batch Normalization
  6. Hyperparameter Tuning
  7. Early Stopping
  8. Cross-Validation
  9. Data Partition-sequential
  10. Ensemble Methods
  11. Data Augmentation
  12. Transfer Learning
  13. Batch Size Selection
  14. Regularization
  15. Gradient Clipping
  1. Model Comparison
  2. Model Interpretability
  3. Cross-Validation
  4. Hyperparameter Tuning
  5. Regularization Techniques
  6. Performance Visualization
  7. Evaluation Metrics
  8. Data Partitioning
  9. Train-Test Split
  10. External Validation
  1. Performance Metrics
  2. Documentation and Reporting
  3. Edge Deployment
  4. Model Retraining and Updating
  5. Containerization
  6. Model Health Monitoring
  7. Error Analysis
  8. Model Drift
  9. Prediction Logging
  10. Concept Drift Detection
  11. Model Serialization
  12. Model Versioning
  13. Cloud Deployment
  14. Serverless Computing
  15. A/B Testing
  16. Streamlit
  17. Bias and Fairness Assessment
  18. Security Considerations
  19. Documentation and API Documentation
  20. Model Monitoring and Maintenance
  21. Feedback Collection
  22. Continuous Integration and Deployment (CI/CD)
  23. Alerting and Notification
  24. Data Drift Monitoring
  25. Web APIs - Flask, FastAPI, etc.
  26. Model Registry
  27. Monitoring and Logging
  1. End User Machine
  2. Mobile
ML Workflow Beginner - Architecture
  • Element belongs to model
  • Element not belongs to model
Feature Store

Feature Store
(Online / Offline)

Data Sources

Data Sources

Data Warehouse

Data Warehouse/ Data Lake

Data Pre Processing & Feature Engineering

EDA, Data Pre Processing & Feature Engineering

Model Selection

Model Selection

Model Training & Hyper Parameter Tuning

Model Training & Hyper Parameter Tuning

Model Evaluation

Model Evaluation

Model Deployment

Model Deployment

End User Device

End User Device

Model Registry

Model Registry