Event
Ph.D. Dissertation Defense: Mohamed Ali
Monday, December 9, 2024
10:00 a.m.
AJC 5104
Rachel Chang
301 405 8268
rachel53@umd.edu
Title: Computer Vision-Guided Robotics for Industrial Automation: Innovations in Depth Imaging and Multi-Tasking Artificial Intelligence Models for Agrifood Processing Solutions
Committee members:
Dr. Yang Tao, Chair
Dr. Jenna Mueller
Dr. Giuliano Scarcelli
Dr. Dongyi Wang
Dr. John Aloimonos, Dean's Representative
Abstract:
The automation of industrial food processing is significantly advanced through the integration of computer vision and robotics, especially for handling complex and non-uniform items. This dissertation explores three primary innovations to improve the automation of front-end loading and packaging tasks for agricultural products.
First, an active dual line-laser scanning system is presented for high-resolution depth imaging of piled items without conveyor movement, overcoming the limitations of traditional depth sensors and achieving sub-millimeter precision. This method improves the depth reconstruction of overlapping piled objects like Chesapeake Blue crabs and White Button mushrooms. Second, the CrabFormer model, a multitasking transformer architecture, is introduced for RGB-D instance segmentation and pose estimation. The dual-patch Swin-T feature extractor applies self-attention mechanism between RGB-D data to learn visual and geometric features. Trained and tested on custom datasets, CrabFormer surpasses existing models in segmentation and keypoint detection Average Precision (AP) and Recall (AR), effectively identifying crabs within complex piles for front-end processing automation. The third component is a vision-guided robotic system for automated mushroom packaging and till weight optimization. This system incorporates real-time in-motion imaging, weighing, instance segmentation, and highspeed delta robot equipped with an electro-pneumatic end effector to handle mushrooms delicately. It optimizes weight distribution across mushroom tills, significantly reducing overfilling error rates and increasing packaging efficiency. The adaptive robotic handling adjusts to variations in mushroom size and weight, maintaining product quality while meeting stringent weight requirements.
Finally, this dissertation explores the ethical considerations surrounding the use of AI and robotics in agricultural automation. As these technologies reshape the labor dynamics in food processing, ethical frameworks are essential to address workforce displacement, agriculture robot accountability, and equitable access to automation benefits. By fostering responsible innovation, this work aims to guide the development of ethical, socially aware robotics applications in agriculture. Together, these contributions propel industrial food processing automation by integrating high resolution depth imaging, advanced computer vision, and robotic handling for irregular agricultural products, paving the way for future research in bioengineering and agricultural robotics.
