Date
Model
Contributors
#Params
Input Length
Score (Average)
GovRep (R1/R2/RL)
SumScr (R1/R2/RL)
QMSum (R1/R2/RL)
Qspr (F1)
Nrtv (F1)
QALT (EM-T/H)
CNLI (EM)
04/27/2022
LongT5 XL
LongT5
3B
16K
41.89
54.7/28.2/30.2
35.8/9.6/21.1
34.9/11.8/23.5
53.1
29.3
46.0/42.1
88.2
04/28/2022
LongT5 Large
LongT5
770M
16K
40.47
54.2/27.8/29.8
35.6/9.2/21.2
35.1/12.0/23.3
52.3
27.2
40.6/38.6
87.3
04/28/2022
LongT5 Base
LongT5
220M
16K
38.22
53.5/27.3/29.3
34.8/9.6/21.1
33.9/11.0/22.8
46.6
23.0
37.9/36.6
85.6
03/14/2022
UL2
Google Research
20B
2K
37.87
53.6/26.1/28.8
32.9/7.8/19.4
31.1/8.5/20.4
37.6
24.2
45.8/40.7
88.7
07/31/2022
BART-large SLED
Ivgi et al.
406M
16K
37.39
58.0/26.9/27.6
33.8/8.0/18.5
32.1/10.2/21.0
46.3
23.6
33.6/33.7
87.0
01/01/2022
LED Base
SCROLLS team
162M
16K
29.16
56.2/26.6/28.8
24.2/4.5/15.4
25.1/6.7/18.8
26.6
18.5
25.8/25.4
71.5
01/01/2022
BART Base
SCROLLS team
139M
1K
29.01
47.9/18.6/22.7
27.2/4.9/16.7
30.2/8.7/20.7
26.3
15.4
26.0/25.9
77.4
01/07/2022
Naive
SCROLLS team
-
-
19.35
45.3/17.9/20.8
19.6/1.8/11.0
14.2/2.0/9.3
3.4
1.5
25.2/26.1
66.0

Click here for a downloadable version of the leaderboard.