Visual Search App Correctly Identifies Objects in the Blink of an Eye

Have you ever passed a dog on the street and wondered the breed, spotted a flower on a hike and wanted to know the species, or just wanted a quicker way to look up restaurant and product reviews? If so, SmartLens is the app you’ve been waiting for. Developed by 18-year-old Michael Royzen, SmartLens turns your phone camera into a search box that is able to correctly identify more than 90 percent of the objects it is pointed at. Its specialties include identifying books, packaged products, animals, and differentiating between different species of flora.

What makes this app unique is the fact that it operates almost entirely offline, making it much faster than cloud-based models. There are more than 17,000 recognizable items built into the app itself, all running through a series of convolutional neural networks―a digital process that simulates the way neural networks work within the brain when identifying objects in the visual field. The entire search and identification process happens on-device within a matter of seconds. However, this also makes the app a huge download at 500 MB. An internet connection is only needed for product recognition since it would be impossible to store every available product within the app.

After identifying an object, the app offers suggestions for what you might want to do next. If it identifies a product, it will provide purchase information. If it identifies a business, it will show reviews. For objects such as animals, plants, food, or landmarks, it includes the Wikipedia description.

“While [the app] isn’t perfect, it’s far better than Google Lens (which can’t recognize basic things such as chairs and bell peppers),” said Royzen in an interview with geekwire.com. “My goal is to get SmartLens’ success rate as close to 100 percent as possible. I have designed a new version of its model architecture which will further increase its speed and accuracy. In the future, customers will be able to correct the app if it is wrong and, with their consent, upload the correct image for future model training.”

As impressive as creating a visual identification app in his spare time was for a high school student, this is actually Royzen’s seventh app. He has been developing software since he was 11 years old. Unsurprisingly, his next stop will be the Turing Scholars program for computer science undergraduates at the University of Texas at Austin in the fall.

Eye on Innovation is a monthly look at new technology, products, and trends.