Robots are rapidly evolving from simple instruments for repetitive tasks to increasingly sophisticated machines capable of performing challenging operations in our daily environment. As they make their way out of custom-made workspaces in factories, intelligent algorithms are needed to enable robots to autonomously execute complex tasks in unstructured environments. The achievement of such algorithms requires interweaving and further development of the concepts and methods originated in various disciplines, namely control theory, motion planning, formal methods, and artificial intelligence. In this talk, I present our recent works that address the challenges in automatic control generation for robotic systems in unstructured environments. The frameworks integrate formal methods, motion planning techniques, and stochastic control concepts to allow a robot with complex continuous dynamics to achieve a high-level task in uncertainty settings.