Advertisement for the field of study such as: Automation technology, cybernetics, aerospace engineering, mechanical engineering, mechatronics, physics, control engineering, software engineering, computer science, robotics or comparable.
Reliable and precise perception is essential for robot-based bin picking and assembly, especially for bulk goods and object manipulation with robotic hands. The focus here is robust multi-camera fusion, accurate object detection in cluttered scenes (bulk goods), and multi-modal tracking that combines camera data with tactile feedback. Using deep learning, multi-view geometry, and simulation via digital twins, the goal is a real-time, deployable perception stack for an industrial robot arm using a robotic hand as an end effector.
What you will do
* Literature Review: Survey multi-camera sensor fusion, object detection under heavy clutter/occlusion, and multi-modal (vision + tactile) tracking in robotics and manufacturing.
Framework Development: Set up the digital twin in Isaac Sim; build scene generators for clutter, occlusions, and varying lighting/reflectivity. Specify dataset strategy (augmentation/domain randomization) and an automated evaluation pipeline. Calibrate and time-synchronize multi-camera systems with tactile sensors; define data schemas and ROS 2/Isaac ROS interfaces.
* Algorithm Implementation: Develop multi-camera fusion for 3D localization (triangulation, multi-view association) using e.g. EKF/UKF. Build multi-modal tracking that fuses object tracking with tactile signals. Optimize for real time (TensorRT, pruning/quantization). Implement robustness strategies for occlusions, small/reflective objects, and provide uncertainty estimates.
* Simulation and Testing: Generate diverse scenarios in Isaac Sim with domain randomization; Evaluate calibration error, FPS/latency, and resource usage. Run ablations on fusion strategies, number/placement of cameras, and tactile on/off; study sim-to-real transfer.
What you bring to the table
* Enrolled student
* High motivation and initiative
* Very good English language skills
* Experience with Isaac Sim / Lab
* Experience in computer vision and deep learning is an advantage
* Independent, responsible, and structured working style
What you can expect
* Pleasant working atmosphere
* interesting and industry-relevant tasks
* dynamic, interdisciplinary team
* Insights into the future topic of automation
* Good technical equipment
We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability. Remuneration according to the general works agreement for employing assistant staff.
With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future.
Interested? Apply online now. We look forward to getting to know you!
Ms. Jennifer Leppich
Recruiting
+49 711 970-1415
Fraunhofer Institute for Manufacturing Engineering and Automation IPA
Requisition Number: 81682 Application Deadline: