In a world increasingly concerned about serious risks from artificial intelligence, policymakers trying to mitigate those risks find themselves turning more and more to a new class of professionals: AI evaluators, who audit systems on behalf of the companies that create them.
kAmr2=:7@C?:2 :D 2E E96 7@C67C@?E @7 6>3C24:?8 E96 6G2=F2E@CD — 2?5 8C2AA=:?8 H:E9 D@>6 @7 E96 AC@3=6>D E96J C2:D6[ :?4=F5:?8 4@?7=:4ED @7 :?E6C6DE 2?5 E96 5:77:4F=EJ @7 2F5:E:?8 86?6C2=\AFCA@D6 E649?@=@8J]k^Am
kAmv@G] v2G:? }6HD@> @? $6AE] h D:8?65 k2 9C67lQ9EEADi^^42=>2EE6CD]5:8:E2=56>@4C24J]@C8^3:==D^420a_ada_ae_D3g`bQm@?6 =2H 6DE23=:D9:?8 DE2?52C5Dk^2m 7@C “:?56A6?56?E G6C:7:42E:@? @C82?:K2E:@?D” E92E H@F=5 6>A=@J 6G2=F2E@CD 2?5 2?@E96C k2 9C67lQ9EEADi^^42=>2EE6CD]5:8:E2=56>@4C24J]@C8^3:==D^420a_ada_ae_23`c_dQm4C62E:?8 2 C68:DECJ @7 6G2=F2E@CDk^2m] w6 2=D@ k2 9C67lQ9EEADi^^HHH]8@G]42]8@G^a_ae^_h^ab^8@G6C?@C\?6HD@>\2??@F?46D\H@C=5\=625:?8\6IA6CED\E@\56=:G6C\@?\9:D\2:\6I64FE:G6\@C56C\:?4=F5:?8\25G2?4:?8\4C62E:@?\@7\2\<:==\DH:E49^Qm2DD6>3=65 2 8C@FA @7 6IA6CEDk^2m H9@ 2C6 E@ C6A@CE 324< :? }@G6>36C E@ C64@>>6?5 H96E96C E@ C6BF:C6 6G2=F2E@CD E@ 36 6>365565 :?D:56 4@>A2?:6D 56G6=@A:?8 E96 >@DE A@H6C7F= px DJDE6>D 2?5 H96E96C E@ D6E DE2?52C5D @? H92E 4@F?ED 2D 2? 256BF2E6 px 2F5:E[ k2 9C67lQ9EEADi^^42=>2EE6CD]@C8^Qmr2=|2EE6CDk^2m C6A@CED]k^Am
kAm%96 >@G6D 4@>6 2>:5 2? :?E6?D:7J:?8 7@4FD @? 6G2=F2E@CD H:E9:? E96 px 64@DJDE6> :ED6=7] ~? $6AE] `g[ >@C6 E92? a__ px C6D62C496CD 2?5 6G2=F2E@CD D:8?65 2 k2 9C67lQ9EEADi^^2:6G2=F2E@C7@CF>]@C8^:?:E:2E:G6D^6>365565\6G2=F2E:@?\=6EE6CQm=6EE6Ck^2m :? 72G@C @7 DE2?52C5D 7@C :?56A6?56?E 6G2=F2E@CDj D:8?2E@C:6D :?4=F565 2 7@C>6C ~A6?px H9:DE=63=@H6C 2?5 E96 9625 @7 E96 &?:E65 }2E:@?D’ x?56A6?56?E x?E6C?2E:@?2= $4:6?E:7:4 !2?6=]k^Am
kAm(96? 2 k2 9C67lQ9EEADi^^HHH]:?56A6?56?E6G2=D]@C8^Qm?6H 8C@FA 7@4FD65 @? px 6G2=F2E:@?k^2m 82E96C65 &?:E65 }2E:@?D @77:4:2=D :? }6H *@C< :? $6AE6>36C a_ae 7@C :ED @77:4:2= =2F?49[ :E 49@D6 2 r2=:7@C?:2 DE2E6 D6?2E@C[ y6CCJ |4}6C?6J @7 $E@4<E@?[ E@ 8:G6 E96 4=@D:?8 C6>2C<D] |4}6C?6J[ 2 s6>@4C2E H9@ 2FE9@C65 E96 G6C:7:42E:@? DE2?52C5D =2H[ 42==65 @? ?2E:@?D 2?5 25G2?465 px 4@>A2?:6D =:<6 p?E9C@A:4 2?5 v@@8=6 E@ 4C62E6 DE2?52C5D 4@>>:EE66D — :? A2CE E@ 96=A 6?DFC6 px 6G2=F2E@CD 42? AC@A6C=J 5@ E96:C ;@3D]k^Am
kAm“(6 42? AFE E96D6 4@>A2?:6D @? ?@E:46[” |4}6C?6J D2:5 2E E96 6G6?E[ “E92E E96J’C6 8@:?8 E@ 36 6G2=F2E65 3J 7@=<D E92E F?56CDE2?5 E96 AC@46DD[ E92E F?56CDE2?5 E96 E649?@=@8J[ 2?5 H:== 36 EC2?DA2C6?E 2?5 9@=5 E96> 244@F?E23=6 D@ E92E E96J 4C62E6 D276 AC@5F4ED[ 2?5 E96J 5@?’E D6?5 @FE E96D6 E9:?8D :?E@ E96 H:=5 E92E 42? 42FD6 92G@4 H:E9 32?<:?8[ H:E9 @FC :?7C2DECF4EFC6 2?5 D@ @?]”k^Am
kAmp? :?E6C?2E:@?2= ;@:?E DE2E6>6?E E92E k2 9C67lQ9EEADi^^G2=E:@?6FG@DE@]7:^6?^\^2\42==\7@C\4@?EC@=\@7\7C@?E:6C\2:\>@56=DQm42==D 7@C :?56A6?56?E px 6G2=F2E:@?Dk^2m 82C?6C65 DFAA@CE 7C@> ?62C=J b_ 4@F?EC:6D[ 244@C5:?8 E@ E96 u:??:D9 |:?:DECJ @7 u@C6:8? p772:CD]k^Am
kAmtG2=F2E@CD 2C6 4@>:?8 E@ E96 7@C6 2=@?8D:56 D6C:@FD px :?4:56?ED] %9:D J62C[ px 286?ED[ H9:=6 36:?8 E6DE65 7@C E96:C 23:=:EJ E@ 924< :?E@ 4@>AFE6C DJDE6>D[ 42CC:65 @FE 2 D6C:6D @7 9:89\AC@7:=6 2EE24<D 282:?DE 3FD:?6DD 2?5 8@G6C?>6?E H63D:E6D] %96 rt~D @7 =625:?8 px 4@>A2?:6D DF3D6BF6?E=J 4@>>:EE65 E@ 6G2=F2E:@?D @7 E96:C >@56=D 3J :?56A6?56?E E9:C5\A2CEJ 2F5:E@CD] p 7@C>6C 6>A=@J66 @7 @?6 DF49 4@>A2?J[ p?E9C@A:4[ 25565 E@ >@F?E:?8 4@?46C?D H96? 96 A@DE65 @?=:?6 23@FE E96 k2 9C67lQ9EEADi^^HHH]HD;]4@>^E649^2:^2?E9C@A:4\C6D62C496C\BF:ED\@G6C\@FE\@7\4@?EC@=\2:\762CD\f_f3feagn627D06?23=65l72=D6QmA@DD:3:=:EJ E92E px >@56=D 4@F=5 <:== 2== 9F>2?Dk^2m]k^Am
kAmtG6? 367@C6 E9@D6 :?4:56?ED[ E96C6’D 366? 2 8C@H:?8 4@?D6?DFD 2>@?8 =2H>2<6CD :? E96 A2DE 76H J62CD E92E 56G6=@A6CD D9@F=5 E6DE px 367@C6 C6=62D6[ D2:5 pDD6>3=J>6>36C #636442 q2F6C\z292?[ 2 s6>@4C2E 7C@> $2? #2>@?] u@C 6I2>A=6[ =2DE J62C[ E6DE:?8 C6BF:C6>6?ED H6C6 :?4=F565 :? 2 k2 9C67lQ9EEADi^^42=>2EE6CD]@C8^64@?@>J^E649?@=@8J^a_ad^_h^2:\C68F=2E:@?\@?\9@=5\282:?^Qm3:== E@ 96=A AC6G6?E 5:D4C:>:?2E@CJ px 564:D:@?Dk^2m 2?5 =68:D=2E:@? :?E6?565 k2 9C67lQ9EEADi^^42=>2EE6CD]@C8^64@?@>J^E649?@=@8J^a_ad^`a^?6H\2:\C68F=2E:@?^QmE@ DE@A px 7C@> 42FD:?8 42E2DEC@A9:4 6G6?EDk^2m] %96 AC@A@D2=D 3F:=E @? 42==D 7@C px E6DE:?8 2?5 >@?:E@C:?8 k2 9C67lQ9EEADi^^G6?EFC6362E]4@>^3FD:?6DD^H92E\2=8@C:E9>\2F5:E:?8\DE2CEFAD\?665\E@\DF44665QmDEC6E49:?8 324< E96 36EE6C A2CE @7 2 564256k^2m] #@F89=J E9C66 @FE @7 7@FC r2=:7@C?:2?D D2:5 E9:D DAC:?8 E92E E96 8@G6C?>6?E D9@F=5 C6BF:C6 E6DE:?8 @7 25G2?465 px >@56=D[ 244@C5:?8 E@ 2 k2 9C67lQ9EEADi^^42C?68:66?5@H>6?E]@C8^C6D62C49^a_ae^_h^a_ae\42C?68:6\42=:7@C?:2\2:\DFCG6JQmr2C?68:6 t?5@H>6?E r2=:7@C?:2 DFCG6Jk^2m]k^Am
kAm}@E 2== E96 2EE6?E:@? @? px 6G2=F2E@CD 92D 366? A@D:E:G6] k^Am
kAmq2F6C\z292?[ H9@ H2D 369:?5 3@E9 @7 E96 2F5:E@C 3:==D }6HD@> D:8?65[ A@:?E65 @FE E92E :?56A6?56?E 6G2=F2E@CD D:8? 4@?EC24ED H:E9 E96 E649 4@>A2?:6D H9@D6 AC@5F4ED E96J 6G2=F2E6[ 2?5 E92E 42? A@D6 2 4@?7=:4E @7 :?E6C6DE]k^Am
kAm“x’> 962C:?8 7C@> 6G2=F2E@CD[ ‘(6 92G6 E@ 36 42C67F=[ 3642FD6 H6 H2?E E96> E@ =6E FD 324< :?[’” D96 D2:5] “$@ H92E 2C6 J@F 2G@:5:?8 D2J:?8 :? @C56C E@ 4@?E:?F6 E@ 82:? 2446DD 2?5 4@?EC24EDn”k^Am
kAmr2C@=:?6 $:686=\$:?89[ 2E E96 (2D9:?8E@?[ s]r] E9:?< E2?< E96 u656C2E:@? @7 p>6C:42? $4:6?E:DED[ 28C665[ D2J:?8 px 2F5:E C68:>6D CF? E96 C:D< @7 C6A62E:?8 >:DE2<6D :? E96 DECF4EFC6 @7 E96 3@?5\C2E:?8 3FD:?6DD] q67@C6 E96 a__g 7:?2?4:2= 4C:D:D[ E96 D6==6CD @7 3@?5D 4@F=5 67764E:G6=J D9@A 7@C E96 C2E:?8 E96J H2?E65] %96? 42>6 E96 4C2D9] $:686= $:?89 E9:?<D 4@>A2?:6D 2?5 8@G6C?>6?ED D9@F=5 A@@= >@?6J E@ A2J 6G2=F2E@CD :?DE625 @7 px =23D A2J:?8 6G2=F2E@CD 5:C64E=J[ 2? :562 E92E px =23 p?E9C@A:4 92D 2=D@ k2 9C67lQ9EEADi^^HHH]2?E9C@A:4]4@>^?6HD^2446?EFC6\6>365565\6G2=F2E:@?Qm6?5@CD65k^2m] k^Am
kAm%96 px C6D62C496CD H9@ C646?E=J D:8?65 E96 =6EE6C 42==:?8 7@C :?5FDECJ DE2?52C5D 7@C 2F5:E@CD D92C6 q2F6C\z292? 2?5 $:686= $:?89’D 4@?46C?D] %96J 2C6 FC8:?8 25G2?465 px 4@>A2?:6D E@ C6=J @? 6G2=F2E@CD H9@ >2:?E2:? 7F== 65:E@C:2= 4@?EC@= 2?5 H9@ >62?:?87F==J 5:D4=@D6 2?5 >:E:82E6 4@?7=:4ED @7 :?E6C6DE] %96J 2=D@ DF886DE65 4@>A2?:6D 8:G6 2446DD E@ >F=E:A=6 6G2=F2E:@? 8C@FAD[ 2== C646:G:?8 E96 D2>6 =6G6= @7 2446DD 2D E96:C @H? D276EJ 6>A=@J66D]k^Am
kAmp?@E96C C:D< :D E92E px 4@>A2?:6D >2J E6DE >@56=D 2D 2 A6C7@C>2E:G6 6I6C4:D6[ q2F6C\z292? D2:5 — 2D 2 >@G6 E@ C62DDFC6 E96 AF3=:4[ H:E9@FE 2 D:?46C6 56E6C>:?2E:@? E@ F?4@G6C AC@3=6>D] $EC@?8 =682= C6BF:C6>6?ED 42? AC6G6?E A6C7@C>2E:G6 E6DE:?8[ D96 25565] k^Am
kAmqFE =682= C6BF:C6>6?ED C2:D6 E96:C @H? AC@3=6>D[ D2:5 |2EE ~’$92F89?6DDJ[ 2 7@C>6C $E2E6 s6A2CE>6?E 2?5 4@?8C6DD:@?2= DE2776C ?@H 2E E96 r6?E6C 7@C s6>@4C24J 2?5 %649?@=@8J[ 2 5:8:E2= C:89ED 25G@424J 8C@FA]k^Am
kAm(96? 2? 2DD6DD>6?E :D 5@?6 D:>A=J E@ >66E 2 =682= C6BF:C6>6?E[ @C82?:K2E:@?D 2C6 >@C6 =:<6=J E@ 5@ E96 >:?:>F> E92E E96J ?665 E@ 5@ :? @C56C E@ 4@>A=J[ 96 2?5 9:D 4@==628F6D C646?E=J 4@?4=F565 :? 2 k2 9C67lQ9EEADi^^45E]@C8^:?D:89ED^86EE:?8\E9:C5\A2CEJ\2:\2DD6DD>6?E\C:89E^QmD6E @7 px 6G2=F2E:@? 8F:56=:?6D 7@C A@=:4J>2<6CDk^2mk^Am
kAm“xE 42? >2<6 :E 92C56C 7@C 2DD6DD@CD E@ 86E E96 3FJ\:? E96J ?665 E@ >2<6 566A6C 492?86D :? 4@>A2?:6D[” ~’$92F89?6DDJ D2:5]k^Am
kAm$:686= $:?89 D2:5 E92E 6G2=F2E@CD D9@F=5 D92C6 E96 C6DF=ED @7 E96:C H@C< E@ 2==@H @E96C 6G2=F2E@CD E@ ;F586 :ED BF2=:EJ[ E9FD 6?4@FC28:?8 >@C6 E9@C@F89 C6G:6H]k^Am
kAm%96 8F:56=:?6D 7C@> ~’$92F89?6DDJ 2?5 9:D 4@==628F6D H2C? E92E AC@A6C=J 6G2=F2E:?8 px >@56=D C6BF:C6D 2 4@?D:56C23=6 C2?86 @7 D<:==D[ :?4=F5:?8 6IA6CED :? AC:G24J[ >6?E2= 962=E9[ 4J36CD64FC:EJ[ 3:2D @FEAFED 2?5 A9JD:42= D276EJ] ~? E@A @7 E92E[ 2F5:ED D9@F=5 2=D@ 6I2>:?6 :?46?E:G6 DECF4EFC6D :?D:56 px 4@>A2?:6D :? 255:E:@? E@ E96 E649?@=@8J :ED6=7[ E96 8F:56=:?6D D2J]k^Am
kAmx? E96 7FEFC6[ 2D E96J C67:?6 E96:C 2AAC@249 E@ px 6G2=F2E:@?[ r2=:7@C?:2 @77:4:2=D >2J 7:?5 E96>D6=G6D 4@==23@C2E:?8 H:E9 :?E6C?2E:@?2= A2CE?6CD] %96 4@?DF=2E6D @C 6>32DD:6D @7 ?2E:@?D D@>6E:>6D AC@>@E6 2?5 4@@C5:?2E6 E96:C E649 A@=:4:6D :? $24C2>6?E@[ 2?5 :? a_ac[ r2=:7@C?:2 =2H>2<6CD k2 9C67lQ9EEADi^^42=>2EE6CD]@C8^64@?@>J^E649?@=@8J^a_ac^_d^42\6F\2:\C68F=2E:@?^QmD@F89E E@ 2=:8? DE2E6 C68F=2E:@?D H:E9 E96 tFC@A62? &?:@?k^2m :? @C56C E@ AC@E64E A6@A=6 7C@> 2CE:7:4:2= :?E6==:86?46] x? pF8FDE[ r2=:7@C?:2 2=D@ DJ?465 :>A=6>6?E2E:@? @7 =2HD C6BF:C:?8 H2E6C>2C<D 7@C px\86?6C2E65 4@?E6?E H:E9 E96 tFC@A62? &?:@?]k^Am
kAmq2F6C\z292? D2:5 D96 DFAA@CED E96 :562 @7 H@C<:?8 H:E9 >@C6 ?2E:@?D[ :? A2CE 3642FD6 C68F=2E:@? :? @?6 ;FC:D5:4E:@? A2G6D E96 H2J 7@C CF=6D :? 2?@E96C]k^Am
kAm“x 5@ E9:?< 4@==23@C2E:@? 2?5 7:?5:?8 H2JD E@ AFD9 E96 6?G6=@A6 E@86E96C :D 96=A7F=[” D96 D2:5]k^Am
kAmp 4@2=:E:@? @7 >:5\D:K65 4@F?EC:6D H@C<:?8 E@86E96C H:E9 r2=:7@C?:2 4@F=5 A@@= E96:C 64@?@>:4 :?7=F6?46 E@ 4@?G:?46 E96 =2C86DE px 4@>A2?:6D E@ 7@==@H E96 <:?5 @7 D276EJ 8F:56=:?6D =2:5 @FE :? E96 :?E6C?2E:@?2= ;@:?E DE2E6>6?E 42==:?8 7@C :?56A6?56?E px 6G2=F2E:@?D[ D2:5 r2?25:2? 2>32DD25@C E@ E96 &?:E65 }2E:@?D s2G:5 {2>6EE:]k^Am
kAm!2D: #2;2=2[ 56AFEJ >:?:DE6C 7@C 7@C6:8? 2772:CD @7 u:?=2?5[ D2:5 r2=:7@C?:2 H@C<:?8 E@86E96C H:E9 D>2== ?2E:@?D “42? A=2J 2 <6J C@=6” :? 255C6DD:?8 72DE\56G6=@A:?8 px C:D<D]k^Am
kAm~’$92F89?6DDJ D2:5 E92E 2? :?E6C?2E:@?2= 7C2>6H@C< 7@C 4@>>6C4:2= px DJDE6> 6G2=F2E:@? :D 8@:?8 E@ 36 C62==J 92C5 E@ 4C62E6 H:E9@FE 2 5@>6DE:4 7C2>6H@C< :? E96 &?:E65 $E2E6D 7:CDE]k^Am
kAm%96 u#~}%xt# p4E[ 2 3:== :? r@?8C6DD E92E H@F=5 6?D9C:?6 E6DE:?8 C6BF:C6>6?ED :?E@ 7656C2= =2H[ 92D DFAA@CE :? r@?8C6DD 7C@> r2=:7@C?:2 #6A] y2J ~36C?@=E6[ 2 #6AF3=:42?] r2=:7@C?:2 s6>@4C2E #6A] %65 {:6F[ H9@ D6CG6D 2D 4@\492:C @7 E96 w@FD6 px 42F4FD[ k2 9C67lQ9EEADi^^HHH]A@=:E:4@]4@>^=:G6\FA52E6D^a_ae^_h^`e^4@?8C6DD^56>D\<66A\42==:?8\7@C\2:\24E:@?\_`_fhb`eQm42==65 7@C 2 4@>>:EE66 G@E6k^2m @? E96 3:== :? $6AE6>36C a_ae]k^Am
kAm“x 42? :>28:?6 D@>6 @7 E96 E9:?8D E92E r2=:7@C?:2 :D 5@:?8 :?7=F6?4:?8 9@H r@?8C6DD E9:?<D 23@FE E96D6 :DDF6D 2?5 E92E 364@>:?8 D@>6E9:?8 3:886C[” 96 D2:5] k^Am
kAmk2 9C67lQ9EEADi^^42=>2EE6CD]@C8^64@?@>J^E649?@=@8J^a_ae^`_^6G2=F2E@CD\2C6\DFAA@D65\E@\<66A\2:\7C@>\<:==:?8\FD\2==\?@\AC6DDFC6^Qmk6>m%9:D DE@CJk^6>mk^2mk6>m H2D AC@5F465 3Jk^6>m k2 9C67lQ9EEADi^^42=>2EE6CD]@C8^Qmk6>mr2=|2EE6CDk^6>mk^2mk6>m 2?5 C6G:6H65 2?5 5:DEC:3FE65 3Jk^6>m k2 9C67lQ9EEADi^^9F3D]=2^"_b<=8$#_Q C6=lQ?@7@==@HQmk6>m$E24<6Ck^6>mk^2mk6>m]k^6>mk^Am
(0) comments
Welcome to the discussion.
Log In
Keep it Clean. Please avoid obscene, vulgar, lewd, racist or sexually-oriented language.
PLEASE TURN OFF YOUR CAPS LOCK.
Don't Threaten. Threats of harming another person will not be tolerated.
Be Truthful. Don't knowingly lie about anyone or anything.
Be Nice. No racism, sexism or any sort of -ism that is degrading to another person.
Be Proactive. Use the 'Report' link on each comment to let us know of abusive posts.
Share with Us. We'd love to hear eyewitness accounts, the history behind an article.