北邮人智-音频感知技术团队用户6625
002-Baseline
输入“/”快速插入内容
002-Baseline
002-Baseline
用户6625用户66252023年8月21日修改
Dataset
Name | CNCVS | NCMMSC-Record | NCMMSC-Vlog | Single100 |
Source | | haitian | Collect from bilibili | bilibili |
Usage | Train | Train, Test | Train, Test | Train, Test |
Speaker | 2557 | 23 | 20 | 1 |
Duration | 300 h | ≈ 20 h | ≈ 20 h | 100h |
Split | | Develope(1/2+1/6), Test(1/3) | Develope(1/2+1/6), Test(1/3) | Train:Valid:Test=8:1:1 |
海天录制的数据集称为Record，诗意从b站收集的数据称为Vlog，两者统称NCMMSC
另外从b站的单个up主【波士顿圆脸】收集了100小时的数据，称为Single100
Sumup
CNCVS Pretrain Experiments
| Model 0 | Model 1 |
Pretrain model | None | Model 0 |
Training data | CNCVS < 4s | CNCVS |
CER on CNCVS | / | 60.07% |
CER on Record-Vlog | / | 76.35% |
CER on Single100 | / | 78.61% |
Log dir | /work104/cchen/VSR/auto_avsr/outputs/2023-08-13/22-00-12/tblog/cncvs_4s/ | /work104/cchen/VSR/auto_avsr/outputs/2023-08-14/12-17-41/tblog/cncvs_4s_30s/version_0 |
Fig |
Challenge Tracks Experiments
Pretrain
| Model 0 | Model 1 |
Pretrain model | None | None |
Training data | NCMMSC < 4s | Single100 < 4s |
CER on Record-Vlog | | |
CER on Single100 | | 63% on <4s valid set 77% on testset |
Log dir | /work101/cchen/code/auto_avsr/outputs/2023-08-18/20-03-50/tblog/sncmmsc_4s/version_0/ | /work102/cchen/code/auto_avsr/outputs/2023-08-18/13-58-03/tblog/single100_4s/version_0/ |
Fig |
Train
| Model 0 | Model 1 | Model 2 | Model 3 |
Pretrain | NCMMSC <4s | CNCVS | single100 <4s | CNCVS |
Training data | NCMMSC | NCMMSC | Single100 | Single100 |