LCD control by Hand Gesture

The Artificial Intelligence Learning (AI) + Machine Learning Kit

Available Now

Instruction and resources

GitHub

Discord

Project Overview

In this project, we build on the finger detection system to create a practical screen control interface. By enhancing how hand landmark data is processed and utilized, we’ve evolved from basic finger counting to a robust system capable of sending precise commands to external displays. The original finger detection project focused on identifying the number of extended fingers using MediaPipe hand landmarks, providing a solid framework for accurate gesture recognition and real-time analysis. These advancements take that foundation further, enabling seamless and intuitive screen control.

Hardware

The following components will be prepared in the kit for this project:

• A breadboard

• A STEPico microcontroller

• Multiple dupont wires

• A ST7789 LCD display

• A USB cable

Breadboard, STEPico, ST7789 display, jumper wires, and USB cable

Tutorial

The key advancement in this project lies in data processing and output formatting. Instead of simply counting extended fingers, we now track each finger's state individually and convert this information into binary signals. This modification enables our system to interface with external devices through serial communication, making it suitable for screen control applications.

The Learning Part

Output Processing

• Individual finger state tracking

• Binary signal conversion

• Structured data formatting for serial communication

System Integration

• Serial communication protocol implementation

• Hardware interface configuration

Project data flow

Hardware

The display for this and following projects is ST7789. This display is a type of thin-film-transistor liquid-crystal display (TFT-LCD) controller driver developed by Sitronix.

A ST7789 display

To wire your STEPico to your ST7789, follow the steps to connect them on breadboard:

1. Power Connections:

• Connect the VCC pin of the display to the 3.3V power pin on STEPico.

• Connect the GND pin of the display to any GND pin on the STEPico.

1. SPI Interface:

• Connect the SCL pin on the display to GP18.

• Connect the SDA pin on the display to GP19.

1. Control Pins:

• Connect the RES pin on the display to GP16.

• Connect the DC pin to GP21.

• Connect the CS pin to GP17.

1. Backlight:

• Connect the BL pin to GP20.

System Dependencies and Library Integration

To implement hardware control and display management, various libraries are needed:

from machine import Pin, SPI
import framebuf
import st7789
from sys import stdin
import uselect

Here's a brief overview of the required libraries:

• machine: Provides Pin and SPI for hardware control.

• framebuf: Manages display buffers for rendering.

• st7789: Offers display-specific commands.

• sys (stdin) and uselect: Facilitate data reception.

Note: The ST7789 driver isn't built into MicroPython by default. Use compatible external libraries like Russ Hughes’ fork of the st7789py_mpy module for enhanced performance and flexibility. Access it here:

https://github.com/russhughes/st7789py_mpy/blob/master/lib/st7789py.py

After you complete the project, it looks like following:

Video Walkthrough