RMIS Leaderboard

We report an overall RMIS score across all datasets, where the higher the score is, the better the model is!

We warmly welcome fellow researchers to test their models on the RMIS benchmark! If you would like to showcase your model on the leaderboard, please provide a minimum inference code and contact us at jab22@mails.tsinghua.edu.cn

Model Performances on the RMIS Benchmark

Model	Variant	Size	RMIS	Anomaly Detection							Fault Diagnosis
				DCASE						Mean	IICA	IIEE	WTPG	MaFaulDa		SDUST		UMGED				PU		Mean
				20	21	22	23	24	25	Mean	IICA	IIEE	WTPG	sound	vib	bearing	gear	sound	vib	vol	cur	vib	cur	Mean
Wav2Vec 2.0	base	95M	53.27	64.76	55.14	55.77	55.69	52.76	55.69	56.64	46.77	96.61	60.81	60.72	78.34	54.12	95.01	9.08	9.34	11.93	15.58	67.00	43.42	49.90
Wav2Vec 2.0	large	315M	50.72	65.61	55.23	54.44	48.09	50.51	33.98	51.31	45.49	72.48	78.33	62.83	87.35	49.98	97.04	10.12	8.40	9.18	12.50	69.24	48.86	50.14
Wav2Vec 2.0	1B	962M	52.46	65.50	55.22	55.81	54.12	51.83	51.60	55.68	44.23	94.97	84.84	49.40	77.08	49.95	96.72	8.39	10.89	8.49	9.93	64.59	40.69	49.24
Wav2Vec 2.0	2B	2.16B	53.71	65.49	55.47	56.35	55.47	53.91	54.53	56.87	42.09	78.81	84.49	59.96	84.14	50.31	98.24	8.37	12.06	10.80	11.13	72.67	44.07	50.55
Whisper	tiny	7.6M	50.03	62.85	54.42	53.72	54.09	52.53	54.01	55.27	40.36	55.16	71.86	44.40	76.92	51.17	96.99	6.71	9.52	9.48	12.51	66.00	41.12	44.78
Whisper	base	19.8M	52.36	65.66	54.89	56.12	56.90	53.70	56.14	57.24	40.10	77.24	83.95	36.35	80.82	51.30	96.90	8.98	9.71	9.02	10.21	69.95	42.70	47.48
Whisper	small	87M	51.40	64.02	54.44	54.85	55.50	53.79	54.67	56.21	41.12	70.21	83.55	38.72	76.75	49.23	95.84	8.63	10.50	8.68	9.89	68.89	43.57	46.58
Whisper	medium	306M	51.63	65.41	54.76	55.58	56.02	54.13	55.21	56.85	41.28	66.84	85.62	37.31	76.66	48.93	96.11	9.29	11.79	8.09	9.61	67.65	44.02	46.40
Whisper	large-v3	635M	51.78	62.27	53.65	54.51	53.97	52.53	54.54	55.25	47.57	70.44	87.24	59.32	73.36	50.85	94.08	9.14	10.10	8.34	8.05	68.80	40.68	48.31
PaSST	base	86M	57.69	66.60	56.84	57.58	59.56	57.64	57.76	59.33	60.50	97.56	92.69	59.30	88.59	61.86	97.11	7.16	9.80	10.64	15.87	74.92	52.62	56.05
AudioMAE	base	85M	51.61	63.87	54.86	56.34	57.96	55.30	56.82	57.52	58.88	53.89	64.02	47.48	80.95	59.74	98.18	6.91	10.93	10.01	11.59	56.02	35.54	45.70
BEATs	iter3	90M	57.30	74.20	60.50	58.95	63.31	56.93	59.09	62.16	77.93	68.17	70.57	52.87	84.45	62.37	98.23	5.98	10.32	10.95	17.23	74.18	48.42	52.44
OpenBEATs	base	90M	55.67	70.86	58.60	57.45	62.48	56.10	57.32	60.47	77.87	70.93	63.60	45.50	83.39	57.31	96.51	5.47	9.51	11.09	13.78	76.96	49.50	50.88
OpenBEATs	large	310M	56.95	72.78	59.93	58.29	63.29	57.32	58.03	61.61	75.18	82.86	62.33	42.69	79.37	70.65	96.86	6.91	9.66	12.63	12.48	78.57	49.67	52.30
EAT	base30	88M	57.62	72.69	58.03	58.84	59.35	56.48	58.65	60.67	64.48	96.65	61.28	68.32	91.90	64.03	95.46	7.89	9.39	12.49	18.87	68.33	50.38	54.57
EAT	large10	310M	58.03	73.81	57.06	57.94	60.29	57.78	60.34	61.20	65.80	94.06	68.91	58.47	88.42	62.64	96.32	9.57	11.60	11.22	15.54	75.74	54.73	54.85
CED	tiny	5.5M	55.36	66.99	56.17	56.75	60.09	56.40	58.40	59.15	47.93	74.23	89.43	53.77	84.56	59.02	96.06	8.09	12.96	10.31	12.65	71.50	49.95	51.57
CED	mini	9.6M	55.95	67.48	56.35	56.59	60.05	57.44	58.26	59.36	50.21	84.26	89.26	56.95	84.88	59.24	95.50	8.36	11.66	10.13	12.43	71.50	48.56	52.53
CED	small	22M	56.01	67.50	56.65	56.87	60.76	57.88	58.15	59.63	50.49	80.23	91.20	54.67	84.67	58.00	96.12	8.05	12.15	10.26	13.14	72.89	49.08	52.38
CED	base	86M	56.71	67.60	56.67	56.95	60.99	57.89	58.45	59.76	52.26	90.08	89.20	57.64	85.95	60.20	95.84	8.12	11.57	10.03	13.63	72.14	50.91	53.66
DaSheng	base	85M	56.37	69.25	57.28	56.02	60.95	56.39	55.82	59.28	74.42	99.10	46.84	63.53	92.06	61.37	98.30	5.64	4.45	11.74	20.08	74.93	42.42	53.45
DaSheng	0.6B	630M	55.43	68.35	56.76	55.35	59.85	56.34	55.41	58.68	75.14	99.25	45.41	57.26	91.11	58.23	97.59	5.27	4.28	11.69	14.68	75.78	42.63	52.18
DaSheng	1.2B	1130M	55.91	69.52	57.06	55.82	61.14	56.32	56.01	59.31	73.18	99.00	48.89	58.16	90.79	55.50	97.99	5.42	4.52	11.76	17.32	75.99	44.11	52.51
MERT	v1-95M	95M	52.96	65.54	55.40	54.68	54.46	53.37	52.74	56.03	58.59	98.07	54.93	42.47	77.55	57.92	98.87	9.26	4.42	11.55	16.38	75.47	43.01	49.88
MERT	v1-330M	315M	56.70	68.79	56.87	55.50	58.34	55.17	55.65	58.39	64.11	99.85	69.02	66.88	92.38	61.71	99.19	5.54	3.92	15.02	18.57	72.97	46.12	55.02
MuQ	msd	333M	53.65	66.91	56.75	56.04	57.69	55.64	56.09	58.19	62.23	95.33	37.98	41.22	88.08	60.67	96.38	6.11	5.72	15.07	9.00	78.57	42.20	49.12
Qwen2-Audio	-	637M	54.73	67.32	55.38	56.46	57.19	56.07	56.27	58.12	45.59	73.56	89.91	59.63	83.35	60.13	97.12	7.63	10.26	9.17	10.63	73.17	47.46	51.35
Qwen2.5-Omni	-	640M	55.40	70.94	57.39	57.20	59.14	56.83	57.05	59.76	45.49	66.56	90.73	53.52	86.21	60.49	97.60	9.02	11.35	9.25	11.87	74.84	46.74	51.05
Audio Flamingo 3	-	637M	55.12	67.02	55.06	57.08	57.52	56.76	56.55	58.33	47.41	88.57	92.41	40.85	87.20	58.27	98.09	7.69	11.56	9.30	10.21	73.37	49.77	51.90
MiDaShengLM	-	630M	53.65	67.10	55.46	56.10	58.70	56.15	56.03	58.26	63.78	59.63	86.39	35.69	76.94	68.31	98.25	7.75	15.54	9.20	7.29	67.47	41.28	49.04
Time-MoE	base	113M	49.03	55.43	51.83	53.72	51.38	50.35	50.58	52.22	56.84	77.21	76.31	47.83	73.75	53.92	92.52	5.29	3.81	12.71	6.52	56.67	32.52	45.84
Sundial	base	116M	50.32	63.87	54.50	53.15	49.06	51.84	52.17	54.10	50.53	98.41	55.57	15.56	59.94	45.01	86.57	5.22	4.54	37.30	26.10	68.73	51.52	46.54
PEEMD	-	-	46.16	58.84	53.25	53.32	53.88	53.63	50.63	53.93	35.40	66.41	61.29	51.23	54.30	30.45	53.14	10.85	10.55	14.58	10.69	63.64	36.49	38.39
TFPred	wc1	3.9M	41.88	58.41	53.22	51.39	51.08	50.66	51.14	52.65	35.35	77.15	24.91	10.94	36.30	26.29	56.56	9.71	9.53	22.48	10.80	51.52	33.02	31.12
LiConvFormer	-	320K	43.99	61.65	53.35	51.96	49.50	51.72	50.62	53.13	33.13	75.10	34.08	13.56	47.15	34.00	71.24	7.52	6.78	25.68	11.92	56.60	36.23	34.85
BearLLM	-	204K	45.75	64.12	53.12	48.95	44.04	51.96	50.47	52.11	26.29	66.25	65.76	45.54	48.99	44.11	77.84	12.66	9.66	11.33	5.59	65.34	32.83	39.40
RotLLM	-	770K	47.70	62.91	53.55	53.80	53.63	51.16	50.94	54.33	56.98	85.82	48.33	15.65	38.65	60.10	92.24	6.89	8.13	9.36	5.00	69.79	37.02	41.07
CoWS	-	6.31M	51.52	59.98	53.93	53.12	52.39	52.07	50.03	53.59	44.72	74.40	86.45	80.43	76.17	48.40	92.96	10.68	4.79	21.04	9.29	59.79	33.80	49.46
ECHO	tiny	5.5M	60.67	70.43	59.01	58.86	63.92	55.77	56.55	60.76	90.31	100.00	65.81	58.84	92.26	76.70	98.78	12.90	21.34	19.92	25.77	74.21	50.87	60.59
ECHO	small	22M	60.31	72.50	60.20	59.13	64.25	56.76	57.45	61.71	90.34	99.95	60.45	61.81	93.18	69.08	98.54	14.17	16.96	14.90	18.34	75.70	52.37	58.91
FISHER	tiny	5.5M	61.12	70.86	58.76	56.40	58.62	53.64	56.37	59.11	85.31	98.72	83.84	75.79	92.58	71.81	99.20	11.90	23.58	21.10	28.24	72.36	56.21	63.13
FISHER	mini	10M	61.43	70.19	58.40	57.62	61.07	54.59	55.75	59.60	86.02	99.05	84.74	75.74	93.20	70.09	98.55	13.69	20.52	22.95	29.97	72.01	55.73	63.25
FISHER	small	22M	62.23	71.04	59.48	59.64	62.63	55.62	58.46	61.15	90.23	98.96	86.77	72.61	95.86	76.35	99.08	12.72	18.35	17.42	25.52	74.90	54.29	63.31

■ Speech Encoders

■ Audio Encoders

■ Music Encoders

■ LALM Audio Encoders

■ Time Series Encoders

■ Signal Encoders

■ Our Model (FISHER)