EcoSnap
ecosnap.mp4
Recycle your plastic better with Artificial Intelligence
EcoSnap tells you how and where to recycle your items from a simple picture, with advice tailored to your location. We built this product in a week for Ben's Bites AI Hackathon.
You can support this project (and many others) through GitHub Sponsors!
❤️
Made by Alyssa X & Leo. Read more about how we built this here.
Table of contents
Features
Installation
You can deploy to Vercel directly by clicking here.
Important: Make sure to update the environment variable for NEXT_PUBLIC_MODEL_URL in the .env file, and set it to an absolute URL where you host the model.json (make sure to include the other shard bin files alongside the JSON).
The AI Model
Data
The model was trained on image examples of the 7 different resin codes, the data for this can be found in ml/seven_plastics
. It is a combination of the following Kaggle Dataset and images collected by the authors and contributors.
Training
The final model was trained using TensorFlow's EfficientNet implementation, the model weights were frozen for transfer learning, so the model could learn the resin codes faster! The model was trained in Python
on a GPU-powered machine, for faster training! You can find the training script in ml/train.py
and try it for yourself, there you will see that different meta architectures and parameters were experimented with before arriving at the final model.
Prediction
To predict the plastic resin code, the model had to be integrated with the front end app for real-time results, to do this we had to convert the model in a way that was compatible with TensorFlow.js. We used Web Workers to prevent the main thread from being block while running the prediction in the client.
The app passes the image Tensor onto the model that then gives a probability for each of the plastic resin codes, the one with the highest probability gets shown to the user, along with bespoke advice!
Feedback
Training a specific model is hard, the model always gets things wrong. So if it does, we give the user an opportunity to tell us what the right code was! This benefits in several ways:
- The user gets the information they need on how to recycle their item
- We can see how the model is performing in production
- We get new data (if the user lets us) to train the model with and improve it for everyone
While we implemented the front end for the feedback loop, we ended up not connecting it to the backend as it added complexity and cost, and we wanted the app to be very lightweight and running entirely on the client. We'd also have to communicate clearly to the user how exactly their images would be used, and set up either an opt-in or opt-out system, which felt a bit cumbersome.
Credit
- Kaggle Dataset - for the plastic codes
- Collletttivo - for the Mattone font
- Stubborn - for some of the illustrations
- Unsplash - for the images
Libraries used
- Tensorflow - for training the model and doing the prediction
- React Camera Pro - for the camera
Feel free to reach out to us at [email protected], to Alyssa or Leo directly if you have any questions or feedback! Hope you find this useful