Towards a Vision-Based Mobile Manipulator for Autonomous Chess Gameplay

Abstract

With the rise of robotic arms in both industrial and research applications, a growing need is observed for autonomous robotic arm applications. This thesis aims to provide an example case of this need and also to showcase the possibility and limitations of vision-based solutions, specifically in automating chess. The focus is on developing a modular system that is able to autonomously recognize chessboard, detect and manipulate chess pieces. The modular design allows for further exploration into autonomous mobile manipulators. The key components include chessboard recognition using fiducial markers to facilitate accurate chessboard recognition and utilizing image processing techniques like segmentation, absolute difference matching, and perspective warping to analyze and extract meaningful information. By mounting a camera above the chessboard, it enables the detection algorithm to accurately capture and analyze the most important information about the environment to determine the current state of the game. Using this information, human move detection is enabled. Then, a custom protocol is utilized to communicate between the detection algorithm and the chess engine, encapsulating information about the game state changes within the system. The chess engine serves the purpose of game analysis and provides legal moves for the robot manipulator to execute. Manipulation happens through careful motion planning and execution, ensuring the safety of the robot and its environment. Extensive evaluation proves that the system demonstrates high accuracy and success rates for piece manipulation and move detection.