I am a fourth year undergraduate student at UC Berkeley advised by Professor Ken Goldberg.
I am interested in building perception systems for language-guided robot interaction and manipulation.
POGS: Persistent Object Gaussian Splat for Tracking Human and Robot Manipulation of Irregularly Shaped Objects
Justin Yu*, Kush Hari*, Karim El-Refai*, Arnav Dalal, Justin Kerr, Chung Min Kim, Richard Cheng, Muhammad Zubair Irshad, Ken Goldberg
*Equal contribution
ICRA 2025 (Under Review)
Website (WIP)
POGS is a system that embeds semantics, self- supervised visual features, and object grouping features into a compact representation which enables online and persistent object tracking and manipulation by dynamically updating the POGS as objects move, without the need for scene recapture or retraining and only using a single stereo camera for tracking.
LEGS: Incrementally Building Room-Scale Language-Embedded Gaussian Splats with a Mobile Robot
Justin Yu*, Kush Hari*, Kishore Srinivas*, Karim El-Refai, Adam Rashid, Chung Min Kim, Justin Kerr, Richard Cheng, Ashwin Balakrishna, Thomas Kollar, Ken Goldberg
*Equal contribution
IROS 2024 (Oral Presentation) arXiv / Website
LEGS provides detailed 3D scene representation that encodes both appearance and semantics in a unified representation while being able to be trained online as a robot traverses its environment to enable localization of open-vocabulary object queries.
SWAG: Storytelling With Action Guidance
Zeeshan Patel*, Karim El-Refai*, Jonathan Pei*, Tianle Li
*Equal contribution
EMNLP Findings 2024
arXiv
SWAG is a novel approach to storytelling with LLMs which reduces story writing to a search problem through a two-model feedback loop. SWAG can substantially outperform previous end-to-end story generation techniques when evaluated by GPT-4 and through human evaluation, and our SWAG pipeline using only open-source models surpasses GPT-3.5- Turbo.
Automating Deformable Gasket Assembly
Simeon Adebola*, Tara Sadjadpour*, Karim El-Refai*, Will Panitch, Zehan Ma, Roy Lin, Tianshuang Qiu, Shreya Ganti, Charlotte Le, Jaimyn Drake, Ken Goldberg
CASE 2024
*Equal contribution
arXiv