Robotic Hand

The Artificial Intelligence Learning (AI) + Machine Learning Kit

Available Now

Instruction and resources

GitHub

Discord

3D Printable Model

Download

Instruction

Project Overview

In this project, we integrate multiple servo motors with a microcontroller to replicate human hand movements. The primary goal is to control a 3D-printed robotic hand using intuitive hand gestures. These gestures are captured, processed, and translated by the microcontroller into precise commands for the servo motors, enabling them to mimic the natural motion of human hand joints. The system demonstrates the seamless interaction between gesture recognition and robotic actuation, paving the way for innovative applications in robotics and human-machine interaction.

Hardware

The following components will be prepared in the kit for this project:

• A breadboard

• A STEPico microcontroller

• Multiple jumper wires

• 5 SG90 servos

• A USB cable

• 3d printed hand model kit (optional)

Breadboard, STEPico, Servos and accessories, jumper wires, and USB cable

Tutorial

In this project, we will further adjust serial communication to enable it to transmit more data in addition to character and binary representation. At the same time, a new type of hardware, servo, will be added to the system. The servo is also controlled by a microcontroller, which outputs hand-related information through the computer. The microcontroller sends a signal to make the servo operate to a specified angle, thereby simulating the movement of the hand joints.

The Learning Part

Hardware Setup

In this project, we will control five servos to simulate hand gestures, so we need to map the servo signal lines to the corresponding GPIO ports one by one. Here, we choose to use GP16 to GP20 in order from thumb to little finger.

Signal transmission

Power Supply

To facilitate the reception of serial communication data and streamline upcoming calculations, we import the following modules:

import time
import uselect
import math
from sys import stdin
from machine import PWM, Pin

The PWM (Pulse Width Modulation) module in MicroPython is an integral part of its control functions, providing users with the ability to manage devices and signals with precision. The importance of this module lies in its comprehensive capabilities, enabling various applications such as motor control, LED brightness adjustments, and more. By incorporating PWM, there is no need for the installation of additional modules, streamlining the development process and optimizing resource utilization.

Servo Control Setup

To ensure proper coordination between finger joints and servos, we will assign each GPIO port to a specific finger.

servo_thumb = Pin(20)
servo_index = Pin(19)
servo_middle = Pin(18)
servo_ring = Pin(17)
servo_pinky = Pin(16)

Define a PWM variable for each servo as follows: servo_pwm5 = PWM(servo_pinky) through servo_pwm1 = PWM(servo_thumb). The PWM() function creates a PWM object, enabling PWM signal generation on microcontroller pins.

Assembly the Hand

The hand model contains six parts: five fingers and one palm with servo connectors on it.

This is a left-handed model, where the index finger and the ring finger shares the same 3d model.

To assembly the hand, first connect the finger base and the servo arm with screws, the screws could be found in servo's component beg

After connecting the finger and the servo arm, connect the servo arm to the servo.

After connecting all the servo arms and servos, place them onto the connectors on the palm. Be sure to arrange the signal and power wires properly.

Joint Angle Recognition

The first function, finger_angle, calculates the bending angle of a finger based on its metacarpophalangeal (MCP), proximal interphalangeal (PIP), and distal interphalangeal (DIP) joints. It computes the angle by evaluating the positions of the corresponding landmarks and returns it as a numerical value, providing a precise measure of finger flexion.

def finger_angel(landmarks, mcp_index, pip_index, dip_index):
	# Determine if a finger is bent (based on angle)
		angle = calculate_angle(landmarks[mcp_index], landmarks[pip_index], landmarks[dip_index])
		return angle

The second function, count_fingers_states, calculates and stores the bending angles of all five fingers, from the thumb to the pinky, by analyzing the hand’s landmarks. It calls finger_angle for each finger, determining the angles of the relevant joints. The result is a list of five floating-point numbers, each representing the current bending angle of one finger joint. This list can be used to track finger states in real-time, enabling more advanced gesture recognition and interaction.

def count_fingers_states(hand_landmarks):
    landmarks = hand_landmarks.landmark
    finger_states = [
        finger_angle(landmarks, 1, 2, 3),  # Thumb
        finger_angle(landmarks, 0, 5, 6),  # Index finger
        finger_angle(landmarks, 0, 9, 10),  # Middle finger
        finger_angle(landmarks, 0, 13, 14),  # Ring finger
        finger_angle(landmarks, 0, 17, 18)  # Pinky finger
    ]
    return finger_states

These functions allow for real-time tracking of finger joint angles, which is useful for gesture recognition and other interactive applications.

After you complete the project, the result looks like the following:

Video Walkthrough