Smart Vending Machine

Projects | | Links: GitHub

Smart Vending Machine

“We design a Smart Vending Machine based on Raspberry Pi 4B. We can buy things in two ways: Face Login or Android App. Here is a live demo of our product!”

Preface

  • This is the final project of Columbia University EECS E4764 Fall’21 IOT.
  • We are Team #14:
  • If you have any questions, please email me at jl5999@columbia.edu.

Abstract

The Smart Vending Machine has two ways of interaction.

  1. The first way doesn’t require anything in hand during the whole buying because we built in a camera with face recognition and a touch screen with beautiful UI. Customers can buy and pay simply with their face.

  2. Users can also use a smartphone if they don’t want to touch the screen. Any operation like shopping, registering, topping up and uploading a photo for face login, can be easily done in the app.

All user profiles and stock information are stored in a secure and real-time cloud database. Our pursuit is to provide users and maintainers a comfortable using experience.

Here is a Live Demo Video to give you an overview of our Smart Vending Machine.

Motivation

Imagine you are running outside and feel thirsty. Luckily, there’s a vending machine with various drinks. You want to buy a bottle of water but you realize that you didn’t bring your wallet. So sad!

However, if it happens to be our vending machine, congratulations!

You just need to log in to our machine with your lovely face and select the item on the touch screen, and everything is done! By the way, any touch could increase the risk of getting Covid, so we also designed a vending machine app for smartphones to help protect your health.

System

Architecture

The architecture of our Smart Vending Machine is summarized in the figure below. We will describe each component of our system in detail in the next section.

<img src=”/assets/img/projects/IoT/architecture.png” width=100% />

Technical Components

  • Embedded systems
    • Raspberry PI 4 Model B

    • Raspberry PI Camera with 5 Million Pixels

    • Raspberry PI 4-Phase Stepper Motor

    • 320x240 2.8” TFT+Touchscreen

  • Cloud Database

    • Firebase Realtime DB + Storage

  • Software

    • App Interacting with Smart Vending Machine

    • Face Recognition Algorithm based on dlib

Prototype

Smart Vending Machine with Face Recognition
+
Android App

Results

  • Android App

    • We designed an Android App shown as above.
    • As a new user, we can Register and Login to the shopping page.
    • We can TOP UP to buy things in Smart Vending Machine.
    • If we want to enable the Face Recognition function of Smart Vending Machine, we need to upload a profile photo.

  • Cloud Data Base

    • We can manage our Firebase RealtimeDB and Storage.
    • We use RealtimeDB to track Product and User information.
    • We use Storage to save the profile photos of our users.

  • Face Recognition

    • Firstly, we use a Deep Neural Network to detect the face location in one image.
    • Then, we encoding the face to a Tensor.
    • After updating the Dataset and keeping it the same as the Cloud Storage, we can comparing the encoded image with every image in our storage, then choose the one has the largest confidence.
    • Finally, we can print the related username and account information.

  • Example A: Buy things using Android App

  • Example B: Buy things with Face Recoginition

References

  1. Modern Face Recognition with Deep Learning

  2. Recognize and manipulate faces from Python or from the command line with the world’s simplest face recognition library

  3. Firebase Documentation

  4. GUIZERO Documentation

Contact

© 2022 Jiawei Lu. All rights reserved.