[논문분석] E2E Segmenter: Joint Segmenting and Decoding for Long-Form ASR
VAD(Voice Activity Detector)와 Streaming End-to-end (E2E) models for ASR을 통합해 성능 향상
PaperReview
[논문분석] Efficient Streaming LLM for Speech Recognition
ASR에 Decoder를 LLM으로 사용, LoRA finetuning
[논문분석] Real Time Speech Enhancement in the Waveform Domain
CPU에서도 실시간으로 동작하는 Sound Enhancement model
[논문분석] Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter
neural contextual adapter를 활용한 context-biasing
[논문분석] CONTEXTUAL ADAPTERS FOR PERSONALIZED SPEECH RECOGNITION IN NEURAL TRANSDUCERS
neural contextual adapter를 활용한 context-biasing
[논문분석] TREE-CONSTRAINED POINTER GENERATOR FOR END-TO-END CONTEXTUAL SPEECH RECOGNITION
Pointer를 활용한 context-biasing
[논문분석] Conformer: Convolution-augmented Transformer for Speech Recognition
ASR 분야에 핵심 모델 : MHSA + Conv
[논문분석] A Survey of Large Language Models
Pre-training, Adaptation, Utilization, Capability Evaluation 에 대한 내용 탐구
[논문분석] DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation
audio signals + face images + landmarks → 통합으로 conditioning
[논문분석] KAN: Kolmogorov–Arnold Networks
KAT에 기반한 Network algorithm : 중간 과정 의미 해석 + 수식 유도 (MLP blackbox와 대조)
[논문분석] Elucidating the Design Space of Diffusion-Based Generative Models
diffusion model에 대한 Practical한 고찰
[논문분석] Person in Place: Generating Associative Skeleton-Guidance Maps for Human-Object Interaction Image Editing
object에 맞는 사람의 pose를 denoise estimate 하고 diffusion based image editing
[논문분석] Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation
Wav2vec + animatediff - Talking Face generation
[논문분석] Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization
Audio Conditioned Diffusion Models - Talking Face generation
[논문분석] Sound-guided Semantic Video Generation
GAN을 활용한 sound guided video generation, clip의 latent space를 활용
[논문분석] HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis
GAN을 활용해 압축된 표현에서 오디오로 변환, one generator and two discriminators
[스터디] Score-Based Generative Models and Diffusion Models
Score-Based Models 스터디
[논문분석] GraphMAE: Self-Supervised Masked Graph Autoencoders
Self-supervised learning을 통해 graph autoencoders 성능 개선
[논문분석] HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
BERT를 활용한 Audio Hidden-Unit 생성
[논문분석] Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation
Hubert를 활용한 Unit Multilingual S2S translation
[논문분석] AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representation
Audio와 Video의 멀티모달 Translate 모델
[논문분석] Latent Video Diffusion Models for High-Fidelity Long Video Generation
Latent space 에서 diffusion process
[논문분석] IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models
Image prompt를 위한 decoupled cross-attention adapter, text prompt와 병행 사용
[논문분석] Lumiere: A Space-Time Diffusion Model for Video Generation
high quality video generation, Google Research
[스터디] Understanding Convolutions on Graphs
Graph를 처리할 수 있는 Convolution Network
[논문분석] ConsistI2V: Enhancing Visual Consistency for Image-to-Video Generation
첫 프레임 특징을 시간축 self-attention에 주입해 일관성을 끌어올린 Image-to-Video 모델
[논문분석] Latte: Latent Diffusion Transformer for Video Generation
diffusion model에 Transformer 구조 사용, video generation model
[논문분석] FIT: Far-reaching Interleaved Transformers
transformer based Diffusion model
[논문분석] High-Resolution Image Synthesis with Latent Diffusion Models
Multi-model image generation diffusion model
[논문분석] Scalable Diffusion Models with Transformers : DIT
transformer based Diffusion model
[논문분석] Gentron : Delving Deep into Diffusion Transformers for Image and Video Generation
U-Net 대신 transformer를 사용하는, DiT를 발전 시킨 모델
[보고서분석] report : An image deblurring method using improved U‐Net model based on multilayer fusion and attention mechanism
U-Net을 기반으로 하는 image deblurring, low cost
[논문분석] TextCraftor: Your Text Encoder Can be Image Quality Controller
Image generation에서 다양한 Condition을 위한 Finetuning 방법
[논문분석] StoryDiffusion : Consistent Self-Attention for long-range inage and video generation
StoryDiffusion : 텍스트 기반 이야기를 일관성 있는 이미지나 영상으로 표현
[논문분석] Adding Conditional Control to Text-to-Image Diffusion Models
conditioning을 제어할 수 있는 Finetuning 방법 ControlNet을 제안
[논문분석] StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text
ControlNet에서 영감을 얻은 long video generation, Picsart AI Research
[논문분석] VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation
Noise 공유를 통한 프레임 간 연관성 향상
[논문분석] Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets
Stable diffusion의 video generation version
[논문분석] VideoPoet: A Large Language Model for Zero-Shot Video Generation
LLM을 활용한 multimodal diffusion generation
[논문분석] Reuse and Diffuse: Iterative Denoising for Text-to-Video Generation
효율적인 LVDM, Reuse and Diffuse : VidRD
[논문분석] Imagen Video: High Definition Video Generation with Diffusion Models
task를 나눠 quality를 높히자
[논문분석] Video DIffuison Models
Diffusion model의 Video generation, base 논문