Spot It Glasses is a pair of smart reading glasses equipped with a built-in camera. It captures images upon touch and sends them to a mobile companion application. On the app users can select specific words to look up instantly making reading more efficient and accessible.
Demo Video and Screenshots: https://youtu.be/ht6kKNzCL6U
Features
- Instant Word Lookup: Allows users to quickly find the meaning of selected words from captured images.
- Camera Integration: Built-in camera captures text upon user input.
- Mobile Companion App: Seamless connectivity with a smartphone application for text processing.
- Customizable Word Selection: Users can choose specific words for quick reference.
Technologies Used
- Hardware: ESP32 camera board with OV2640 camera module.
- Embedded Programming: Developed using Embedded C for ESP32 board.
- Mobile App: Developed using Android Studio with Java and XML.
- OCR Technology: Ml Kit Library .
- Communication: Bluetooth for data transmission between ESP32 and the mobile app.
Mobile Application Setup
-
- Download or clone this repository:
Alternatively, you can download the ZIP file and extract it.git clone https://github.com/yourusername/spot-it-glasses.git cd spot-it-glasses - Open Android Studio and select Open an Existing Project.
- In the file selection window, navigate to the extracted
spot-it-glassesfolder. - Allow Android Studio to sync Gradle and install necessary dependencies automatically.
- Connect your Android device via USB and enable Developer Mode.
- Click Run (▶) in Android Studio to install and launch the application on your device.
- Alternatively, you can generate an APK by navigating to Build > Build Bundle(s) / APK(s) > Build APK(s) and installing the APK manually.
ESP32 Firmware Setup
- Connect the ESP32 board to your computer using a USB cable.
- Open the Arduino IDE and install the necessary ESP32 board support package.
- Select the appropriate ESP32 board from the Tools > Board menu.
- Open the firmware code from the repository (
esp32_firmwarefolder). - Select the correct port under Tools > Port.
- Click the Upload button to flash the firmware onto the ESP32.
- Restart the ESP32 module after flashing.
- Pair the ESP32 with your mobile application via Bluetooth.
Usage
- Wear the glasses and power them on.
- Tap to capture an image of the text.
- The image is sent to the mobile companion app.
- Select specific words to look up instantly on the app.
Future Improvements
- Enhancing OCR accuracy for better text recognition.
- Adding support for multiple languages.
- Implementing voice feedback for word meanings.
- Expanding compatibility with additional mobile platforms.
- Integrating with a Vision Model: Enhancing text recognition and object detection capabilities using AI-based vision models.
For inquiries or collaborations, reach out at our mail.
This is a prototype project and should not be used as a medical or safety device.
Akshaykumar, Kaviyarasan, Churchill L.
This project is licensed under the Apache License 2.0