Subin Kim
M.S. Student, KAIST
School of Computing
21supersoo [AT] kaist.ac.kr
About
Hello! I am currently a second-year M.S. student in the School of Computing at KAIST advised by Sungjin Ahn. Previously, I completed my B.S. in Computer Science at KAIST, and minored in Industrial and Systems Engineering.
The reason the Industrial Revolution was able to change the world was that it enabled the operation of machines that replaced human physical labor. Chat-GPT is gaining attention because it replaces human intellectual labor and increases work efficiency. I believe that within the next 10 years, physical labor will again be replaced by robots. However, these robots will not be like the heavily human-dependent machines of the Industrial Revolution era. Instead, they will be capable of thinking for themselves, generalizing, and solving various tasks autonomously. To create such robots, I think it is necessary to conduct research on creating the “brain” of a robot.
I believe that to create generalizable robots, two things are necessary. This is also the common factor of models in other fields like Chat-GPT for LLM or Dall-E for vision: Data (=Knowledge/Experience, Input) & Structure (= Brain, Algorithm). Chat-GPT was trained using an enormous amount of data, and generative AIs like Dall-E have shown tremendous performance improvements recently with techniques like diffusion methods. In the robotics field, there is also a project called Open X-Embodiment, where 21 institutions worldwide are gathering data on robots performing 160,266 tasks.
Based on this trend, I want to research two things:
- Although a large amount of data can be collected, it is often inconsistent and noisy. I want to study ways to standardize this data into a single format for training using LLMs.
- I want to research how to enable trained agents to combine what they have learned in new ways to perform new tasks not present in the existing data (Out-Of-Distribution, OOD).
To enable this, I want to make use of the following fields:
- World Models Learning to abstract the general rules of the world and plan the future outcomes through imagination.
- Skill Discovery Using skill is the ability to segment observations into meaningful intervals and recombine them, giving the power to create & adapt to Out-Of-Distribution environments.
- Offline RL Allows agents to learn effective policies from previously collected data without the need for costly and potentially unsafe interactions with the environment.
Education
KAIST MLML Lab Daejeon, Republic of Korea
M.S. in School of Computing Mar. 2023 - Feb. 2025(expected)
KAIST Daejeon, Republic of Korea
B.S. in Computer Science Mar. 2018 - Feb. 2023
Minor in Industrial and Systems engineering
Semiminor in Artificial Intelligence
Publications
* indicates equal contribution.
Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming
{Hany Hamed*, Subin Kim*}, Dongyeong Kim, Jaesik Yoon, Sungjin Ahn
ICML 2024
Playgrounds for abstraction and reasoning
Subin Kim, Prin Phunyaphibarn, Donghyun Ahn, Sundong Kim
nCSI WS @ NeurIPS 2022
Korea Software Congress (KSC) 2022 (Best Paper Award)
Domestic
Test Bed for Abstraction and Reasoning
Subin Kim, Prin Phunyaphibarn, Donghyun Ahn, Sundong Kim
Journal of KIISE, Vol. 51, No. 1, Jan 2024
Analysis of impact on Seoul traffic congestion by Land Use using travel speed and POI
Subin Kim, Sumin Han, Dongman Lee
Korea Software Congress (KSC) 2021
Research Experiences
Cornell PoRTaL(remote) Ithaca, NY, U.S.
Research InternOct. 2024 - Present
Advisor: Sanjiban Choudhury, Arnav Kumar Jain, Gokul Swamy
Hybrid training of Offline diffusion policy & online MBRL (To submit to ICML 2025)
- Investigate how the different image representation effects diffusion policy and world models.
- Investigate how offline training with expert data can enhance the online learning through world models.
- Implement MPPI planner (referred from TD-MPC) with n-step TD reward using the trained reward function and critic of the world model.
- Skill: Pytorch, Docker, Singularity
KAIST MLMLabDaejeon, Republic of Korea
M.S.Mar. 2023 - Feb. 2025 (expected)
Advisor: Sungjin Ahn
Divide-and-conquer strategy for goal-conditioned MBRL agent (ICML 2024)
- Proposed the concept of “strategic dreaming” in pixel-based MBRL, where the agent can leverage the structure of the state space to enable a divide-and-conquer-like strategy.
- Improved prior methods in visually complex and partially observable navigation tasks.
- Published “Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming (ICML 2024)” as first co-author.
- Skill: Pytorch, Docker, Mujoco
KAIST DS Lab Daejeon, Republic of Korea
Research InternJuly. 2022 - Jan. 2023
Advisor: Sundong Kim, Meeyoung Cha
Simplify & extend ARC for Research-optimized dataset (NeurIPS nCSIW 2022)- Built Mini-ARC that extends the original ARC dataset with restricted size.
- Built O2ARC, which is an object-oriented web interface to solve & suggest Mini-ARC.
- Hosted Happy ARC Day including over 40 participants to create and peer review Mini-ARC.
- Published "Playgrounds for Abstraction and Reasoning" in nCSI WS @ NeurIPS 2022.
- Published "Test Bed for Abstraction and Reasoning (Short version)" & Received Best Paper Award in KSC 2022.
- Gave a poster presentation in nCSI WS @ NeurIPS & KSC 2022.
- Skill: HTML, CSS, Javascript, Python, Flask, Django, ML
KAIST DM LabSeoul, Republic of Korea
Research InternDec. 2021 - June. 2022
Advisor: Kijung Shin
- Studied topics related to graph neural networks (GNN)
- Lead to the opportunity of starting "Open Dialogue Generation based on knowledge graph"
- Skill: Python, GNN
KAIST PROSYS LabDaejeon, Republic of Korea
Research InternJune. 2021 - Sep. 2021
Advisor: Kihong Heo
"Tracer: Signature-based Static Analysis for Detecting Recurring Vulnerabilities"(Submitted to ICSE 2022, 2nd author)
- 'Tracer' detects software vulnerabilities by exploiting existing vulnerability patterns.
- Participated as a second author, implementing ML baselines and hunting bugs to use for training data.
- Poster presentation for KAIST PL seminar (Host: Sukyoung Ryu)
- This project was accepted to CCS 2022. However, I did not participate in the later process.
- Skill: Python, C, C++, Pytorch, PL, ML
KAIST CDSN LabDaejeon, Republic of Korea
Research InternJan. 2021 - June. 2021
Advisor: Sumin Han, Dongman Lee
- Participated in predicting traffic flow through spatio-temporal neural networks.
- Submitted "Spatio-Temporal Attention based Conditional Traffic Flow Prediction in Urban Areas" to ECML-PKDD 2021 as 2nd author
- Publishd "Analysis of impact on Seoul traffic congestion by Land Use using travel speed and POI" in KSC 2021 as 1st author
- Skill: Python, Pytorch, Pandas, spatio-temporal prediction
Work Experiences
Bookend Daejeon, Republic of Korea
Software Engineering InternAug. 2022 - Dec. 2022
- Participated in building chrome extension using Chat-GPT with React, openAI API
- Skill: React, React-Native, HTML, CSS, Javascript, MongoDB, ChatGPT API
Bank of Wine Daejeon, Republic of Korea
Software Engineering InternJune. 2022 - Aug. 2022
- Participated in building internal payment system of the company with React, Redux, MongoDB, and AntD
- Skill: React, React-Native, AntD UI, HTML, CSS, Javascript, MongoDB
RedWit Daejeon, Republic of Korea
Software Engineering InternJune. 2021 - Aug. 2021
- Participated in building internal scheduling system of the company with React, Redux, and MongoDB
- Skill: React, React-Native, HTML, CSS, Javascript, MongoDB
Teaching Experiences
Deep Reinforcement Learning and Game AI Daejeon, Republic of Korea
Teaching AssistantMarch. 2024 - June. 2024
Best TA Award
Hana Bank-KAIST Digital Warrior Program Daejeon, Republic of Korea
TA of Computer Science ProjectMarch. 2022 - June. 2022
TA of Data StructureOct. 2021 - Dec. 2021
KAIST Korean for International Students Daejeon, Republic of Korea
Teaching AssistantFeb. 2022 - Dec. 2022
KAIST Korean Camp for International Students Daejeon, Republic of Korea
Teaching AssistantJan. 2022 - Feb. 2022
Miscellaneous
Google ExploreCSR Daejeon, Republic of Korea
Workshop OrganizerSep. 2023 - Aug. 2024
Certified as Korean Red Cross lifeguard Daejeon, Republic of Korea
Jan. 2023 - Present
KAIST swimming team KAORI Daejeon, Republic of Korea
Senior memberJun. 2021 - Present
50m Silver medal in women backstroke (Daejeon Masters)Jun. 2022
50m Bronze medal in women freestyle (Daejeon Masters)Jun. 2022
Projects
NMix Q-learning: Investigating overestimation bias of Q-values
Subin Kim, Kyungwook Nam, Doojin Baek
Deep Reinforcement Learning (Spring 2023) (A+)
Skill: Pytorch, Python, Mujoco, Conda, DeepRL
Improving Directed Greybox Fuzzing using Monte-Carlo Decision Tree and Ant Colony Optimization
Taeeun Kim, Subin Kim, Geonho Koh, and Sungsoo Han
Artificial Intelligence Based Software Engineering (Fall 2021) (A+)
Skill: Python, C, LLVM, PL
Tracer: Signature-based Static Analysis for Detecting Recurring Vulnerabilities
Wooseok Kang, Byoungho Son*, Subin Kim*, Kihong Heo
Submitted to ICSE 2022
Skill: Python, C, C++, Docker, PL
NMix Q-learning: Investigating overestimation bias of Q-values
Subin Kim, Kyungwook Nam, Doojin Baek
Deep Reinforcement Learning (Spring 2023) (A+)
Skill: Pytorch, Python, Mujoco, Conda, DeepRL
Bank Of Wine Internal Payment System
Subin Kim
Bank Of Wine Intern Project summer 2022
Skill: React, React-Native, AntD UI, HTML, CSS, Javascript, MongoDB
Open Dialogue Generation based on knowledge graph
Subin Kim, Jinhyuk Jang, Sihyeon Kim, Duri Lee
Introduction to Artificial Intelligence (Spring 2022) (S)
Skill: Pytorch, Python, Tensorflow, GNN
Improving Directed Greybox Fuzzing using Monte-Carlo Decision Tree and Ant Colony Optimization
Taeeun Kim, Subin Kim, Geonho Koh, and Sungsoo Han
Artificial Intelligence Based Software Engineering (Fall 2021) (A+)
Skill: Python, C, LLVM, PL
Tracer: Signature-based Static Analysis for Detecting Recurring Vulnerabilities
Wooseok Kang, Byoungho Son*, Subin Kim*, Kihong Heo
Submitted to ICSE 2022
Skill: Python, C, C++, Docker, PL
Cheers: Interactive social platform for live-stream baseball game
Subin Kim, Chigon Ryu, Dain Kim, Huikyeong Ann
Artificial Intelligence Based Software Engineering (Fall 2021) (A+)
Skill: React, React-Native, HTML, CSS, Javascript, MongoDB
RedWit Internal Scheduling System
Subin Kim, Chigon Ryu
RedWit Intern Project summer 2021
Skill: React, React-Native, HTML, CSS, Javascript, MongoDB
PINTOS-KAIST: operating system framework for the x86-64 architecture (forked from stanford pintos project)
Subin Kim, Huikyeong Ann
Operating Systems and Lab (Spring 2021) (A)
Lab1 100/100 (avg 90.3), Lab2 118.24/120 (avg 85.08), Lab3 104.17/115 (avg 68.55), Lab4 133/150 (avg 59.78)
Skill: C, Linux OS
KENSV3: template for building prototypes of Ethernet/ARP/IP/TCP
Subin Kim, Huikyeong Ann
Introduction to Computer Networks (Fall 2021) (A)
PA1 7.19/8 (avg 6.57), PA2 7.5/8 (avg 6.62), PA3 6.95/8 (avg 5.18), PA4 9/9 (avg 7.17) + additional score, total 30/30 (avg 24.96)
Skill: C++, Docker, Ethernet/IP/TCP
Appendix
Dr. Strategy: Model-Based Generalist Agents with Strategic DreamingICML 2024
What?
How can we make generalist agents that can reach diverse goals without specific training, with only pixel-level observation?
How?
- Model-Based RL (MBRL) enables the agent to learn its policy within an internal model of the world instead of the real world via planning (Hence, dreaming).
- Divide-and-Conquer strategy improves efficiency by breaking a problem into smaller, more manageable subproblems, solving each independently, and combining their solutions to address the original problem.
- The agent builds a set of Latent Landmarks during exploration, and then utilizes these to navigate large state spaces. With the highway policy, the agent can first learn in the dream to move to a landmark, and from there it tackles the exploration and achievement task in a more focused manner.
Role
- Proposed Focused Sampling which lead to a crucial distinction of success rate from other baselines.
- Lead the visualization of Figure 1 & Figure 2, containing key ideas of the paper.
- Lead the writing of Section 2(Method) & Revised and wrote Section 3(Experiment)
Playgrounds for abstraction and reasoningnCSI WS @ NeurIPS 2022
What?
How can we measure the intelligence of AGI?
How?
- Object-Oriented ARC (O2ARC) enables people to create, evaluate & solve ARC problems with additional primitive yet semantic functions.
- Mini-ARC is a 5x5 compact version of the Abstraction and Reasoning Corpus (ARC), enabling easier level problems for basic AGI models.
Role
- Built Mini-ARC that extends the original ARC dataset with restricted size.
- Built O2ARC, which is an object-oriented web interface to solve & suggest Mini-ARC.
- Hosted Happy ARC Day including over 40 participants to create and peer review Mini-ARC.
- Received Best Paper Award in KSC 2022.
- Gave a poster presentation in nCSI WS @ NeurIPS & KSC 2022.
Vitæ
Full CV in PDF.
-
KAIST Mar. 2023 - Feb. 2025 (expected)M.S. in School of Computing
MLML Lab -
KAIST DS Lab July. 2022 - Jan. 2023Research Intern
Abstraction and Reasoning Corpus -
Bank of Wine June. 2022 - Aug. 2022Software Engineering Intern
Full-stack -
KAIST DM Lab Dec. 2021 - June. 2022Research Intern
Graph Nueral Networks (GNN) -
KAIST PROSYS Lab June. 2021 - Sep. 2021Research Intern
software vulnerability detection -
RedWit June. 2021 - Aug. 2021Software Engineering Intern
Frontend -
KAIST CDSN Lab Jan. 2021 - June. 2021Research Intern
spatio-temporal neural network -
KAIST March. 2018 - Feb. 2023B.S. in Computer Science
-
Sangsan High School Mar. 2015 - Feb. 2018High school
2nd place in Sangsan High Mathematics Competition