[Takeaways] Invited Talk by Li Yi (Assistant Prof. @ Tsinghua) and He Wang (Assistant Prof. @ Peking)

1. Li Yi's Talk

• Title: Interaction-Oriented 4D Visual Understanding and Synthesis
• Takeaways
  • I should persue towards 3D scenes, not only in 2D scene. (I am planning to take CS487 course for this.)
  • -> This will further require the need of 4D deep visual scene understanding. What direction should I take? How can my vision be integrated to this world?
  • Are the robot's ultimate goal humaoid robots? This professor seems to assume and then pursue his work. Is this scalable or generalization? I should think about this more.
  • I am looking forward to check his work Leaf, accepted in CVPR 2023. Interactive object state estimation... I should be aware of that keyword.
  • Why is interaction useful? It may take demonstration, collaboration, simulation in advance. But actually I do not agree with this. I think the interaction terminology definition is different from mine. I should think more on interactive perception and think how to meet the best of both worlds.

Notes

• Interaction-oriented 4D visual understanding and synthesis

1. Static perception to dynamic interaction (Demonstration, Collaboration, Simulation)
2. Demonstration -> 4D dynamic interaction dataset
   1. Capturing diverse interaction data
3. Collaboration (DIY) -> Interaction-oriented 4D visual understanding
   1. Understanding interaction dynamics, semantics and intention
4. Simulation -> Dynamic interaction synthesis
   1. Synthesize realistic human to enrich robot simulators-

• Challenges
  • Persistent understanding in 4D (3D space + 1D time)
  • More than semantics (human attention, kinematics, visual sensoimotor behavior)
  • Interpret scenes and actions in human’s context
  • Scalability and generalizability

2. He Wang's Talk

• Title: Towards 3D-Aware Multi-Function Home Robots
• Takeaways
  • How can the grasp be more task-oriented, or more customized? I should talk with Donguk about this issue.