Jongha Kim

AI Researcher @ Sogang University
Audio-Language Intelligence
LALMs & Robust ASR

📄 Curriculum Vitae (PDF)  ·  📄 Project Portfolio (PDF)

Education

Sogang University, Seoul, Korea · Mar. 2026 – Present M.S. Student, Department of Artificial Intelligence

Intelligent Information Processing Lab (IIP) · Advisor: Prof. Hyung-Min Park

Inha University, Incheon, Korea · Feb. 2019 – Aug. 2025 B.S. in Electronic Engineering

GPA 3.98 / 4.5

Experience

Graduate Researcher · Mar. 2026 – Present Intelligent Information Processing Lab (IIP), Sogang University

  • Robust automatic speech recognition for the medical domain (SCH).
  • Integration of robust ASR with language models.

Research Intern · Jul. 2025 – Feb. 2026 Intelligent Information Processing Lab (IIP), Sogang University

Joined the lab before the M.S. program.

ML Scientist / Embedded Engineer · Mar. 2025 – Jun. 2025 R&D Department, STS Engineering

  • Soft-sensor prediction of water quality indicators from IoT sensor time series.
  • MCU firmware on FreeRTOS for continuous unattended operation.

Research Intern · Jan. 2024 – Dec. 2024 Machine Intelligence Lab, Inha University

  • Video synthesis with diffusion models; scene-graph-conditioned video generation.
  • Long video generation.

Research Intern · Jul. 2022 – Dec. 2022 Intelligence Embedding System Lab (IESL), Inha University

  • Computer vision for robotics and autonomous driving.

Publications

  1. “TASK-LEAF ROUTED MIMO-AUDIO FOR DCASE 2026 TASK 5.” Detection and Classification of Acoustic Scenes and Events (DCASE) 2026 Challenge, Jul. 1, 2026. Ranked 6th out of 11 lightweight systems.
  2. Jang Ji-hye, Lee Young-jun, Heo Ji-won, Kim Jong-ha. “Development of an ROS-based Environmental Perception and Decision-making System for Indoor Autonomous Mobile Robots.” Korean Society of Automotive Engineers (KSAE), Jeju, Korea, Oct. 2022. (Poster)

Projects

Audio-to-Audio Speech Unit Translation (Korean ↔ English) · Sep. 2024 – Present ECE Capstone Design, Inha University

  • Textless speech-to-speech pipeline: mHuBERT with a k-means quantizer (500 units) → Transformer encoder–decoder with language tags → HiFi-GAN unit vocoder conditioned on a speaker d-vector.
  • Extended prior SVO-only work to Korean (SOV): reordered transcripts with Llama 3.1 8B Instruct before resynthesis, so the model aligns by time frame rather than by lexical correspondence.
  • BLEU 42.8 / COMET 0.1009 (AV2AV baseline: 60.1 / 0.1587); output intelligible but unstable in pitch.
  • Setup: Multilingual AIhub + LibriSpeech, Fairseq / PyTorch, NVIDIA A6000 48GB.

Scene Graph to Video Generation with Diffusion · Aug. 2024 – Dec. 2024 Machine Intelligence Lab, Inha University

  • R-GCN scene-graph embedding aligned to a CLIP image encoder by contrastive learning (SGClip), conditioning a latent diffusion model through cross-attention with a time-extended U-Net.
  • Autoregressive long-video module injecting noise only into the first frame.
  • Pipeline fully implemented; results limited by scarce Scene-Graph–video paired data (Action Genome) and weak graph–video alignment.

Water Quality Prediction: Embedded Device and Sensor Analysis · Mar. 2025 – Jun. 2025 R&D Department, STS Engineering

  • IoT device with water quality sensors, RS485 serial, and LTE; FreeRTOS firmware focused on fault handling and reboot recovery.
  • Ensemble separating trend prediction from noise modelling for irregular sensor series.

Competitions

Korean LLM Fine-tuning for Question Answering · Jul. 2024 – Aug. 2024 2024 Inha Artificial Intelligence Challenge (Dacon)

  • QA over Korean economic articles; found LoRA adaptation degraded an already well-aligned base model, so the approach shifted to minimal fine-tuning that preserves base behaviour.

Global Wildfire Detection Challenge · Mar. 2024 6th AI SPARK Challenge · github.com/hytric/Wildfire-detection

  • Satellite-image segmentation with TransUNet and Attention U-Net; ~90% with a single model, improved by ensembling.

Earlier Work

  • Vision-based Autonomous Human-Following Wheeled Mobile Robot — FVE Alpha Project, Inha University (Sep. – Dec. 2022). Led to the KSAE 2022 poster above.
  • Model Ensemble ViT-SSD — Vision Transformer with Single Shot Detection, ECE Deep Learning course project (Nov. – Dec. 2023).
  • Real-time Computer Vision on AWS + Raspberry Pi — Hanium ICT Challenge (Mar. – Aug. 2023).

Teaching

Teaching Assistant · Sep. 2024 – Dec. 2024 Inha University, Dept. of Electronic Engineering

Deep Learning · Introduction to Machine Learning

Awards

  • Academic Excellence Scholarship, Inha University — Mar. 2024
  • Encouragement Prize, Convergence Project 2022-2, Inha University — Dec. 2022
  • Encouragement Award, Winter Break Job Analysis Online Competition, Inha University — Jan. 2023

Entrepreneurship

  • Startup: Product Development and Branding — Gyeonggi Content Agency, 20M KRW funding (Jan. – Jul. 2023).
  • SeTA (Social Entrepreneurship Team Academy) — SKKU / MTA Korea, global entrepreneurship training program (Mar. – Jun. 2022).

Skills

  • Programming — Python, C, C++, Linux
  • Frameworks — PyTorch (Lightning), TensorFlow, Fairseq
  • Certifications — SQLD (SQL Developer), ADSP (Advanced Data Analytics Semi-Professional)