ModelVale is an app designed to bring machine learning to life. Each model is represented by an avatar with a dynamic health bar representing the overall model's performance. For instance, with a model classifying objects (e.g. if there is a picture of a dog, it will predict the label "dog"), the health bar fills when the model correctly classifies an object, decreases when more computation is used to train it, and more. Over time you can see your best performing models and their attributes, and keep training and testing their predictions to keep up their health.
Required Must-have Stories
- Sign in and register with ModelVale account
- User persistence
- Unique avatars to represent each user's model(s)
- Uses CoreML to package models and be able to use them for predictions and retraining
- Health bar of the model to represent overall performance Technical Problem
- Base off of correct to incorrect prediction ratio
- Time slowly decreases health
- Training computational resources drains health
- Larger model size drains health
- Is a comparable, meaningful metric across disparate models (i.e. the health bar is one way to compare how "well" an image classifier is doing vs a object detection model, vs other different types)
- User uploads or provides dataset link to data to test the model on
- User can retrain pretrained model using immediate data
- Photos from camera or camera roll (convolutional networks)
- Other: Sound? Video? Text?
- User can upload a new pretrained model
- Animation of the health bar as it increases or decreases after testing Technical Problem
- Sign out of the app
Optional Nice-to-have Stories
- Background music and improved avatar aesthetics
- Visualize training process and important weights in network
- Models can mutate their structures and create new models
- Genetic algorithm over autoencoder latent layer of source code of models
- Display GPT-3 performance and baseline, non-NN models like linear regression, random forest
- Have multiple avatars at once
- Model leaderboards
- In-game achievements
- See stats about best performing models across time
- This uses a database backend like Parse or Firebase to store preferences
- Push notifications about model performance
Other Possible Technical Problems
- Planning for very large datasets and fetching in chunks to display data
- Use multithreading and progress bars to retrain models in the background