Subin Kim

M.S. Student, KAIST

School of Computing

21supersoo [AT] kaist.ac.kr

About

Hello! I am currently a second-year M.S. student in the School of Computing at KAIST advised by Sungjin Ahn. Previously, I completed my B.S. in Computer Science at KAIST, and minored in Industrial and Systems Engineering.

The reason the Industrial Revolution was able to change the world was that it enabled the operation of machines that replaced human physical labor. Chat-GPT is gaining attention because it replaces human intellectual labor and increases work efficiency. I believe that within the next 10 years, physical labor will again be replaced by robots. However, these robots will not be like the heavily human-dependent machines of the Industrial Revolution era. Instead, they will be capable of thinking for themselves, generalizing, and solving various tasks autonomously. To create such robots, I think it is necessary to conduct research on creating the “brain” of a robot.

I believe that to create generalizable robots, two things are necessary. This is also the common factor of models in other fields like Chat-GPT for LLM or Dall-E for vision: Data (=Knowledge/Experience, Input) & Structure (= Brain, Algorithm). Chat-GPT was trained using an enormous amount of data, and generative AIs like Dall-E have shown tremendous performance improvements recently with techniques like diffusion methods. In the robotics field, there is also a project called Open X-Embodiment, where 21 institutions worldwide are gathering data on robots performing 160,266 tasks.

Based on this trend, I want to research two things:

Although a large amount of data can be collected, it is often inconsistent and noisy. I want to study ways to standardize this data into a single format for training using LLMs.
I want to research how to enable trained agents to combine what they have learned in new ways to perform new tasks not present in the existing data (Out-Of-Distribution, OOD).

To enable this, I want to make use of the following fields:

World Models Learning to abstract the general rules of the world and plan the future outcomes through imagination.
Skill Discovery Using skill is the ability to segment observations into meaningful intervals and recombine them, giving the power to create & adapt to Out-Of-Distribution environments.
Offline RL Allows agents to learn effective policies from previously collected data without the need for costly and potentially unsafe interactions with the environment.

Education

KAIST MLML Lab Daejeon, Republic of Korea

M.S. in School of Computing Mar. 2023 - Feb. 2025(expected)

Advisor: Sungjin Ahn

KAIST Daejeon, Republic of Korea

B.S. in Computer Science Mar. 2018 - Feb. 2023

Minor in Industrial and Systems engineering

Semiminor in Artificial Intelligence

Publications

* indicates equal contribution.

Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming

{Hany Hamed*, Subin Kim*}, Dongyeong Kim, Jaesik Yoon, Sungjin Ahn

ICML 2024

Paper Poster BibTeX

Playgrounds for abstraction and reasoning

Subin Kim, Prin Phunyaphibarn, Donghyun Ahn, Sundong Kim

nCSI WS @ NeurIPS 2022

Korea Software Congress (KSC) 2022 (Best Paper Award)

Paper Poster Code BibTeX

Domestic

Test Bed for Abstraction and Reasoning

Subin Kim, Prin Phunyaphibarn, Donghyun Ahn, Sundong Kim

Journal of KIISE, Vol. 51, No. 1, Jan 2024

Paper

Analysis of impact on Seoul traffic congestion by Land Use using travel speed and POI

Subin Kim, Sumin Han, Dongman Lee

Korea Software Congress (KSC) 2021

Paper

Research Experiences

KAIST MLMLabDaejeon, Republic of Korea

M.S.Mar. 2023 - Feb. 2025 (expected)

Advisor: Sungjin Ahn

Divide-and-conquer strategy for goal-conditioned MBRL agent (ICML 2024)

Proposed the concept of “strategic dreaming” in pixel-based MBRL, where the agent can leverage the structure of the state space to enable a divide-and-conquer-like strategy.
Improved prior methods in visually complex and partially observable navigation tasks.
Published “Dr. Strategy: Model-Based Generalist Agents with Strategic Dreaming (ICML 2024)” as first co-author.
Skill: Pytorch, Docker, Mujoco

KAIST DS Lab Daejeon, Republic of Korea

Research InternJuly. 2022 - Jan. 2023

Advisor: Sundong Kim, Meeyoung Cha

Simplify & extend ARC for Research-optimized dataset (NeurIPS nCSIW 2022)

Built Mini-ARC that extends the original ARC dataset with restricted size.
Built O2ARC, which is an object-oriented web interface to solve & suggest Mini-ARC.
Hosted Happy ARC Day including over 40 participants to create and peer review Mini-ARC.
Published "Playgrounds for Abstraction and Reasoning" in nCSI WS @ NeurIPS 2022.
Published "Test Bed for Abstraction and Reasoning (Short version)" & Received Best Paper Award in KSC 2022.
Gave a poster presentation in nCSI WS @ NeurIPS & KSC 2022.
Skill: HTML, CSS, Javascript, Python, Flask, Django, ML

KAIST DM LabSeoul, Republic of Korea

Research InternDec. 2021 - June. 2022

Advisor: Kijung Shin

Studied topics related to graph neural networks (GNN)
Lead to the opportunity of starting "Open Dialogue Generation based on knowledge graph"
Skill: Python, GNN

KAIST PROSYS LabDaejeon, Republic of Korea

Research InternJune. 2021 - Sep. 2021

Advisor: Kihong Heo

"Tracer: Signature-based Static Analysis for Detecting Recurring Vulnerabilities"(Submitted to ICSE 2022, 2nd author)

'Tracer' detects software vulnerabilities by exploiting existing vulnerability patterns.
Participated as a second author, implementing ML baselines and hunting bugs to use for training data.
Poster presentation for KAIST PL seminar (Host: Sukyoung Ryu)
This project was accepted to CCS 2022. However, I did not participate in the later process.
Skill: Python, C, C++, Pytorch, PL, ML

KAIST CDSN LabDaejeon, Republic of Korea

Research InternJan. 2021 - June. 2021

Advisor: Sumin Han, Dongman Lee

Participated in predicting traffic flow through spatio-temporal neural networks.
Submitted "Spatio-Temporal Attention based Conditional Traffic Flow Prediction in Urban Areas" to ECML-PKDD 2021 as 2nd author
Publishd "Analysis of impact on Seoul traffic congestion by Land Use using travel speed and POI" in KSC 2021 as 1st author
Skill: Python, Pytorch, Pandas, spatio-temporal prediction

Work Experiences

Bookend Daejeon, Republic of Korea

Software Engineering InternAug. 2022 - Dec. 2022

Participated in building chrome extension using Chat-GPT with React, openAI API
Skill: React, React-Native, HTML, CSS, Javascript, MongoDB, ChatGPT API

Bank of Wine Daejeon, Republic of Korea

Software Engineering InternJune. 2022 - Aug. 2022

Participated in building internal payment system of the company with React, Redux, MongoDB, and AntD
Skill: React, React-Native, AntD UI, HTML, CSS, Javascript, MongoDB

RedWit Daejeon, Republic of Korea

Software Engineering InternJune. 2021 - Aug. 2021

Participated in building internal scheduling system of the company with React, Redux, and MongoDB
Skill: React, React-Native, HTML, CSS, Javascript, MongoDB

Teaching Experiences

Deep Reinforcement Learning and Game AI Daejeon, Republic of Korea

Teaching AssistantMarch. 2024 - June. 2024

Best TA Award

Hana Bank-KAIST Digital Warrior Program Daejeon, Republic of Korea

TA of Computer Science ProjectMarch. 2022 - June. 2022

TA of Data StructureOct. 2021 - Dec. 2021

KAIST Korean for International Students Daejeon, Republic of Korea

Teaching AssistantFeb. 2022 - Dec. 2022

KAIST Korean Camp for International Students Daejeon, Republic of Korea

Teaching AssistantJan. 2022 - Feb. 2022

Miscellaneous

Google ExploreCSR Daejeon, Republic of Korea

Workshop OrganizerSep. 2023 - Aug. 2024

Certified as Korean Red Cross lifeguard Daejeon, Republic of Korea

Jan. 2023 - Present

KAIST swimming team KAORI Daejeon, Republic of Korea

Senior memberJun. 2021 - Present

50m Silver medal in women backstroke (Daejeon Masters)Jun. 2022

50m Bronze medal in women freestyle (Daejeon Masters)Jun. 2022

Projects

Selected
All

NMix Q-learning: Investigating overestimation bias of Q-values

Subin Kim, Kyungwook Nam, Doojin Baek

Deep Reinforcement Learning (Spring 2023) (A+)

Skill: Pytorch, Python, Mujoco, Conda, DeepRL

Proposal Paper Slides Video Code

Improving Directed Greybox Fuzzing using Monte-Carlo Decision Tree and Ant Colony Optimization

Taeeun Kim, Subin Kim, Geonho Koh, and Sungsoo Han

Artificial Intelligence Based Software Engineering (Fall 2021) (A+)

Skill: Python, C, LLVM, PL

Paper Slides Code

Tracer: Signature-based Static Analysis for Detecting Recurring Vulnerabilities

Wooseok Kang, Byoungho Son*, Subin Kim*, Kihong Heo

Submitted to ICSE 2022

Skill: Python, C, C++, Docker, PL

Paper Poster

NMix Q-learning: Investigating overestimation bias of Q-values

Subin Kim, Kyungwook Nam, Doojin Baek

Deep Reinforcement Learning (Spring 2023) (A+)

Skill: Pytorch, Python, Mujoco, Conda, DeepRL

Proposal Paper Slides Video Code

Bank Of Wine Internal Payment System

Subin Kim

Bank Of Wine Intern Project summer 2022

Skill: React, React-Native, AntD UI, HTML, CSS, Javascript, MongoDB

Code

Open Dialogue Generation based on knowledge graph

Subin Kim, Jinhyuk Jang, Sihyeon Kim, Duri Lee

Introduction to Artificial Intelligence (Spring 2022) (S)

Skill: Pytorch, Python, Tensorflow, GNN

Proposal Paper Slides Video

Improving Directed Greybox Fuzzing using Monte-Carlo Decision Tree and Ant Colony Optimization

Taeeun Kim, Subin Kim, Geonho Koh, and Sungsoo Han

Artificial Intelligence Based Software Engineering (Fall 2021) (A+)

Skill: Python, C, LLVM, PL

Paper Slides Code

Tracer: Signature-based Static Analysis for Detecting Recurring Vulnerabilities

Wooseok Kang, Byoungho Son*, Subin Kim*, Kihong Heo

Submitted to ICSE 2022

Skill: Python, C, C++, Docker, PL

Paper Poster

Cheers: Interactive social platform for live-stream baseball game

Subin Kim, Chigon Ryu, Dain Kim, Huikyeong Ann

Artificial Intelligence Based Software Engineering (Fall 2021) (A+)

Skill: React, React-Native, HTML, CSS, Javascript, MongoDB

Proposal Paper Slides Video Code Code(Backend)

RedWit Internal Scheduling System

Subin Kim, Chigon Ryu

RedWit Intern Project summer 2021

Skill: React, React-Native, HTML, CSS, Javascript, MongoDB

Video Code

PINTOS-KAIST: operating system framework for the x86-64 architecture (forked from stanford pintos project)

Subin Kim, Huikyeong Ann

Operating Systems and Lab (Spring 2021) (A)

Lab1 100/100 (avg 90.3), Lab2 118.24/120 (avg 85.08), Lab3 104.17/115 (avg 68.55), Lab4 133/150 (avg 59.78)

Skill: C, Linux OS

Proposal Code

KENSV3: template for building prototypes of Ethernet/ARP/IP/TCP

Subin Kim, Huikyeong Ann

Introduction to Computer Networks (Fall 2021) (A)

PA1 7.19/8 (avg 6.57), PA2 7.5/8 (avg 6.62), PA3 6.95/8 (avg 5.18), PA4 9/9 (avg 7.17) + additional score, total 30/30 (avg 24.96)

Skill: C++, Docker, Ethernet/IP/TCP

Proposal Code

Appendix

Dr. Strategy: Model-Based Generalist Agents with Strategic DreamingICML 2024

What?

How can we make generalist agents that can reach diverse goals without specific training, with only pixel-level observation?

How?

Model-Based RL (MBRL) enables the agent to learn its policy within an internal model of the world instead of the real world via planning (Hence, dreaming).
Divide-and-Conquer strategy improves efficiency by breaking a problem into smaller, more manageable subproblems, solving each independently, and combining their solutions to address the original problem.
The agent builds a set of Latent Landmarks during exploration, and then utilizes these to navigate large state spaces. With the highway policy, the agent can first learn in the dream to move to a landmark, and from there it tackles the exploration and achievement task in a more focused manner.

Role

Proposed Focused Sampling which lead to a crucial distinction of success rate from other baselines.
Lead the visualization of Figure 1 & Figure 2, containing key ideas of the paper.
Lead the writing of Section 2(Method) & Revised and wrote Section 3(Experiment)

Playgrounds for abstraction and reasoningnCSI WS @ NeurIPS 2022

What?

How can we measure the intelligence of AGI?

How?

Object-Oriented ARC (O2ARC) enables people to create, evaluate & solve ARC problems with additional primitive yet semantic functions.
Mini-ARC is a 5x5 compact version of the Abstraction and Reasoning Corpus (ARC), enabling easier level problems for basic AGI models.

Role

Built Mini-ARC that extends the original ARC dataset with restricted size.
Built O2ARC, which is an object-oriented web interface to solve & suggest Mini-ARC.
Hosted Happy ARC Day including over 40 participants to create and peer review Mini-ARC.
Received Best Paper Award in KSC 2022.
Gave a poster presentation in nCSI WS @ NeurIPS & KSC 2022.

Vitæ

Full CV in PDF.

KAIST Mar. 2023 - Feb. 2025 (expected)

M.S. in School of Computing
MLML Lab
KAIST DS Lab July. 2022 - Jan. 2023

Research Intern
Abstraction and Reasoning Corpus
Bank of Wine June. 2022 - Aug. 2022

Software Engineering Intern
Full-stack
KAIST DM Lab Dec. 2021 - June. 2022

Research Intern
Graph Nueral Networks (GNN)
KAIST PROSYS Lab June. 2021 - Sep. 2021

Research Intern
software vulnerability detection
RedWit June. 2021 - Aug. 2021

Software Engineering Intern
Frontend
KAIST CDSN Lab Jan. 2021 - June. 2021

Research Intern
spatio-temporal neural network
KAIST March. 2018 - Feb. 2023

B.S. in Computer Science
Sangsan High School Mar. 2015 - Feb. 2018

High school
2nd place in Sangsan High Mathematics Competition