LCD control by Hand Gesture

the Artificial Intelligence Learning (AI) + Machine Learning Kit

Available Now

Instruction and resources

Github link

Discord Link

Project Overview

In this project, we build on the finger detection system to create a practical screen control interface. By enhancing how hand landmark data is processed and utilized, we’ve evolved from basic finger counting to a robust system capable of sending precise commands to external displays. The original finger detection project focused on identifying the number of extended fingers using MediaPipe hand landmarks, providing a solid framework for accurate gesture recognition and real-time analysis. These advancements take that foundation further, enabling seamless and intuitive screen control.

Hardware

The following components will be prepared in the kit for this project:

• A breadboard

• A STEPico microcontroller

• Multiple dupont wires

• A ST7789 LCD display

• A USB-C cable

Tutorial

This project comes with a detailed 80-page tutorial packed with step-by-step instructions and helpful illustrations. It walks you through the assembly process and dives into more advanced topics like how microcontrollers work, coding with MicroPython, and understanding key electronic components and sensors. Whether you're just starting out or looking to deepen your knowledge, this guide makes complex ideas easy to follow and fun to learn.

The Learning Part

Output Processing

• Individual finger state tracking

• Binary signal conversion

• Structured data formatting for serial communication

System Integration

• Serial communication protocol implementation

• Hardware interface configuration

Hardware

The display for this and following projects is ST7789. This display is a type of thin-film-transistor liquid-crystal display (TFT-LCD) controller driver developed by Sitronix.

To wire your STEPico to your ST7789, follow the steps to connect them on breadboard(figure 5.2.3):

1. Power Connections:

• Connect the VCC pin of the display to the 3.3V power pin on STEPico.

• Connect the GND pin of the display to any GND pin on the STEPico.

1. SPI Interface:

• Connect the SCL pin on the display to GP18.

• Connect the DSA pin on the display to GP19.

1. Control Pins:

• Connect the RES pin on the display to GP16.

• Connect the DC pin to GP21.

• Connect the CS pin to GP17.

1. Backlight:

• Connect the BL pin to GP20.

System Dependencies and Library Integration

To implement hardware control and display management, various libraries are needed:

from machine import Pin, SPI
import framebuf
import st7789
from sys import stdin
import uselect

Here's a brief overview of the required libraries:

• machine: Provides Pin and SPI for hardware control.

• framebuf: Manages display buffers for rendering.

• st7789: Offers display-specific commands.

• sys (stdin) and uselect: Facilitate data reception.

Note: The ST7789 driver isn't built into MicroPython by default. Use compatible external libraries like Russ Hughes’ fork of the st7789py_mpy module for enhanced performance and flexibility. Access it here: https://github.com/russhughes/st7789py

If everything is working correctly, a green rectangle will appear in the upper left corner of the screen

After you complete the project, it looks like following: