Robotic Hand

the the Artificial Intelligence Learning (AI) + Machine Learning Kit

Available Now

Instruction and resources

Github link

Discord Link

Project Overview

In this project, we integrate multiple servo motors with a microcontroller to replicate human hand movements. The primary goal is to control a 3D-printed robotic hand using intuitive hand gestures. These gestures are captured, processed, and translated by the microcontroller into precise commands for the servo motors, enabling them to mimic the natural motion of human hand joints. The system demonstrates the seamless interaction between gesture recognition and robotic actuation, paving the way for innovative applications in robotics and human-machine interaction.

Hardware

The following components will be prepared in the kit for this project:

• A breadboard

• A STEPico microcontroller

• Multiple dupont wires

• 5 SG90 servos

• A USB-C cable

• 3d printed hand model kit (optional)

Tutorial

This project comes with a detailed 60-page tutorial packed with step-by-step instructions and helpful illustrations. It walks you through the assembly process and dives into more advanced topics like how microcontrollers work, coding with MicroPython, and understanding key electronic components and sensors. Whether you're just starting out or looking to deepen your knowledge, this guide makes complex ideas easy to follow and fun to learn.

The Learning Part

Hardware Setup

In this project, we will control five servos to simulate hand gestures, so we need to map the servo signal lines to the corresponding GPIO ports one by one. Here, we choose to use GP16 to GP20 in order from thumb to little finger.

To facilitate the reception of serial communication data and streamline upcoming calculations, we import the following modules:

import time
import uselect
import math
from sys import stdin
from machine import PWM, Pin

The PWM (Pulse Width Modulation) module in MicroPython is an integral part of its control functions, providing users with the ability to manage devices and signals with precision. The importance of this module lies in its comprehensive capabilities, enabling various applications such as motor control, LED brightness adjustments, and more. By incorporating PWM, there is no need for the installation of additional modules, streamlining the development process and optimizing resource utilization.

Servo Control Setup

To ensure proper coordination between finger joints and servos, we will assign each GPIO port to a specific finger.

servo_thumb = Pin(20)
servo_index = Pin(19)
servo_middle = Pin(18)
servo_ring = Pin(17)
servo_pinky = Pin(16)

Define a PWM variable for each servo as follows: servo_pwm5 = PWM(servo_pinky) through servo_pwm1 = PWM(servo_thumb). The PWM() function creates a PWM object, enabling PWM signal generation on microcontroller pins.

Joint Angle Recognition

The first function, finger_angle, calculates the bending angle of a finger based on its metacarpophalangeal (MCP), proximal interphalangeal (PIP), and distal interphalangeal (DIP) joints. It computes the angle by evaluating the positions of the corresponding landmarks and returns it as a numerical value, providing a precise measure of finger flexion.

def finger_angel(landmarks, mcp_index, pip_index, dip_index):
	# Determine if a finger is bent (based on angle)
		angle = calculate_angle(landmarks[mcp_index], landmarks[pip_index], landmarks[dip_index])
		return angle

The second function, count_fingers_states, calculates and stores the bending angles of all five fingers, from the thumb to the pinky, by analyzing the hand’s landmarks. It calls finger_angle for each finger, determining the angles of the relevant joints. The result is a list of five floating-point numbers, each representing the current bending angle of one finger joint. This list can be used to track finger states in real-time, enabling more advanced gesture recognition and interaction.

def count_fingers_states(hand_landmarks):
    landmarks = hand_landmarks.landmark
    finger_states = [
        finger_angle(landmarks, 1, 2, 3),  # Thumb
        finger_angle(landmarks, 0, 5, 6),  # Index finger
        finger_angle(landmarks, 0, 9, 10),  # Middle finger
        finger_angle(landmarks, 0, 13, 14),  # Ring finger
        finger_angle(landmarks, 0, 17, 18)  # Pinky finger
    ]
    return finger_states

These functions allow for real-time tracking of finger joint angles, which is useful for gesture recognition and other interactive applications.

After you complete the project, the result looks like the following: