default search action
José Hernández-Orallo
Person information
- affiliation: Polytechnic University of Valencia, Spain
Refine list
refinements active!
zoomed in on ?? of ?? records
view refined list in
export refined list as
showing all ?? records
2020 – today
- 2024
- [j58]Lexin Zhou, Wout Schellaert, Fernando Martínez-Plumed, Yael Moros-Daval, Cèsar Ferri, José Hernández-Orallo:
Larger and more instructable language models become less reliable. Nat. 634(8032): 61-68 (2024) - [c97]Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Seán Ó hÉigeartaigh, Anna Korhonen, José Hernández-Orallo:
Your Prompt Is My Command: On Assessing the Human-Centred Generality of Multimodal Models (Abstract Reprint). AAAI 2024: 22712 - [c96]José Hernández-Orallo:
Caveats and Solutions for Characterising General-Purpose AI. ECAI 2024: 2-9 - [c95]Behzad Mehrbakhsh, Fernando Martínez-Plumed, José Hernández-Orallo:
Distilling the Effects of Language Model Contamination. ECAI 2024: 2298-2305 - [c94]Yael Moros-Daval, Fernando Martínez-Plumed, José Hernández-Orallo:
Language Task Difficulty Prediction Through LLM-Annotated Meta-Features. ECAI 2024: 2434-2441 - [c93]Daniel Romero-Alvarado, José Hernández-Orallo, Fernando Martínez-Plumed:
How Resilient are Language Models to Text Perturbations? IDEAL (1) 2024: 85-96 - [e13]Gabriel Pedroza, Xiaowei Huang, Xin Cynthia Chen, Fabio Arnez, Huáscar Espinoza, José Hernández-Orallo, Mauricio Castillo-Effen, Richard Mallah, John A. McDermid, Andreas Theodorou:
Proceedings of the IJCAI 2024 Workshop on Artificial Intelligence Safety (AISafety 2024) co-located with the 33rd International Joint Conference on Artificial Intelligence (IJCAI 2024), Jeju, Korea, August 4, 2024. CEUR Workshop Proceedings 3856, CEUR-WS.org 2024 [contents] - [i47]Cèsar Ferri, Darío Garigliotti, Brigt Arve Toppe Håvardstun, José Hernández-Orallo, Jan Arne Telle:
When Redundancy Matters: Machine Teaching of Representations. CoRR abs/2401.12711 (2024) - [i46]David Nieves, María José Ramírez-Quintana, Carlos Monserrat, César Ferri, José Hernández-Orallo:
Learning Alternative Ways of Performing a Task. CoRR abs/2404.02579 (2024) - [i45]Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, José Hernández-Orallo, Lewis Hammond, Eric J. Bigelow, Alexander Pan, Lauro Langosco, Tomasz Korbak, Heidi Zhang, Ruiqi Zhong, Seán Ó hÉigeartaigh, Gabriel Recchia, Giulio Corsi, Alan Chan, Markus Anderljung, Lilian Edwards, Yoshua Bengio, Danqi Chen, Samuel Albanie, Tegan Maharaj, Jakob N. Foerster, Florian Tramèr, He He, Atoosa Kasirzadeh, Yejin Choi, David Krueger:
Foundational Challenges in Assuring Alignment and Safety of Large Language Models. CoRR abs/2404.09932 (2024) - [i44]John Burden, Manuel Cebrián, José Hernández-Orallo:
Conversational Complexity for Assessing Risk in Large Language Models. CoRR abs/2409.01247 (2024) - [i43]Lorenzo Pacchiardi, Lucy G. Cheke, José Hernández-Orallo:
100 instances is all you need: predicting the success of a new LLM on unseen data by testing on a few instances. CoRR abs/2409.03563 (2024) - [i42]Lorenzo Pacchiardi, Marko Tesic, Lucy G. Cheke, José Hernández-Orallo:
Leaving the barn door open for Clever Hans: Simple features predict LLM benchmark answers. CoRR abs/2410.11672 (2024) - 2023
- [j57]Fernando Martínez-Plumed, José Hernández-Orallo:
Training Data Scientists Through Project-Based Learning. Rev. Iberoam. de Tecnol. del Aprendiz. 18(3): 295-304 (2023) - [j56]Wout Schellaert, Fernando Martínez-Plumed, Karina Vold, John Burden, Pablo A. M. Casares, Bao Sheng Loe, Roi Reichart, Seán Ó hÉigeartaigh, Anna Korhonen, José Hernández-Orallo:
Your Prompt is My Command: On Assessing the Human-Centred Generality of Multimodal Models. J. Artif. Intell. Res. 77: 377-394 (2023) - [j55]Gonzalo Jaimovitch-López, Cèsar Ferri, José Hernández-Orallo, Fernando Martínez-Plumed, María José Ramírez-Quintana:
Can language models automate data wrangling? Mach. Learn. 112(6): 2053-2082 (2023) - [j54]Manuel Garcia-Piqueras, José Hernández-Orallo:
Heuristic search of optimal machine teaching curricula. Mach. Learn. 112(10): 4049-4080 (2023) - [j53]Radosvet Desislavov, Fernando Martínez-Plumed, José Hernández-Orallo:
Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning. Sustain. Comput. Informatics Syst. 38: 100857 (2023) - [j52]Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew K. Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakas, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartlomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, Cèsar Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodolà, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan J. Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, François Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocon, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse H. Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, José Hernández-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Senel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, María José Ramírez-Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael I. Ivanitskiy, Michael Starritt, Michael Strube, Michal Swedrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T., Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Milkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima (Shammie) Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay V. Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, Ziyi Wu:
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. Trans. Mach. Learn. Res. 2023 (2023) - [c92]Behzad Mehrbakhsh, Fernando Martínez-Plumed, José Hernández-Orallo:
Adversarial Benchmark Evaluation Rectified by Controlling for Difficulty. ECAI 2023: 1696-1703 - [c91]Brigt Arve Toppe Håvardstun, Cèsar Ferri, José Hernández-Orallo, Pekka Parviainen, Jan Arne Telle:
XAI with Machine Teaching When Humans Are (Not) Informed About the Irrelevant Features. ECML/PKDD (3) 2023: 378-393 - [e12]Gabriel Pedroza, Xiaowei Huang, Xin Cynthia Chen, Andreas Theodorou, José Hernández-Orallo, Mauricio Castillo-Effen, Richard Mallah, John A. McDermid:
Proceedings of the Workshop on Artificial Intelligence Safety 2023 (SafeAI 2023) co-located with the Thirty-Seventh AAAI Conference on Artificial Intelligence (AAAI 2023), Washington DC, USA, February 13-14, 2023. CEUR Workshop Proceedings 3381, CEUR-WS.org 2023 [contents] - [e11]Gabriel Pedroza, Xiaowei Huang, Xin Cynthia Chen, Andreas Theodorou, Huáscar Espinoza, Nikolaos Matragkas, José Hernández-Orallo, Mauricio Castillo-Effen, Richard Mallah, John A. McDermid, David M. Bossens, Bettina Könighofer, Sebastian Tschiatschek, Anqi Liu:
Proceedings of the IJCAI-23 Joint Workshop on Artificial Intelligence Safety and Safe Reinforcement Learning (AISafety-SafeRL 2023) co-located with the 32nd International Joint Conference on Artificial Intelligence(IJCAI2023), Macau, China, August 21-22, 2023. CEUR Workshop Proceedings 3505, CEUR-WS.org 2023 [contents] - [i41]Anthony G. Cohn, José Hernández-Orallo:
Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs. CoRR abs/2304.11164 (2023) - [i40]Ryan Burnell, Han Hao, Andrew R. A. Conway, José Hernández-Orallo:
Revealing the structure of language model capabilities. CoRR abs/2306.10062 (2023) - [i39]John Burden, Konstantinos Voudouris, Ryan Burnell, Danaja Rutar, Lucy Cheke, José Hernández-Orallo:
Inferring Capabilities from Task Performance with Bayesian Triangulation. CoRR abs/2309.11975 (2023) - [i38]Lexin Zhou, Pablo Antonio Moreno Casares, Fernando Martínez-Plumed, John Burden, Ryan Burnell, Lucy Cheke, Cèsar Ferri, Alexandru Marcoci, Behzad Mehrbakhsh, Yael Moros-Daval, Seán Ó hÉigeartaigh, Danaja Rutar, Wout Schellaert, Konstantinos Voudouris, José Hernández-Orallo:
Predictable Artificial Intelligence. CoRR abs/2310.06167 (2023) - [i37]Ross Gruetzemacher, Alan Chan, Kevin Frazier, Christy Manning, Stepán Los, James Fox, José Hernández-Orallo, John Burden, Matija Franklin, Clíodhna Ní Ghuidhir, Mark M. Bailey, Daniel Eth, Toby D. Pilditch, Kyle A. Kilian:
An International Consortium for Evaluations of Societal-Scale Risks from Advanced AI. CoRR abs/2310.14455 (2023) - [i36]Xiting Wang, Liming Jiang, José Hernández-Orallo, Luning Sun, David Stillwell, Fang Luo, Xing Xie:
Evaluating General-Purpose AI with Psychometrics. CoRR abs/2310.16379 (2023) - [i35]Konstantinos Voudouris, Ibrahim Alhas, Wout Schellaert, Matthew Crosby, Joel Holmes, John Burden, Niharika Chaubey, Niall Donnelly, Matishalin Patel, Marta Halina, José Hernández-Orallo, Lucy G. Cheke:
Animal-AI 3: What's New & Why You Should Care. CoRR abs/2312.11414 (2023) - 2022
- [j51]Tijl De Bie, Luc De Raedt, José Hernández-Orallo, Holger H. Hoos, Padhraic Smyth, Christopher K. I. Williams:
Automating data science. Commun. ACM 65(3): 76-87 (2022) - [c90]Pablo Antonio Moreno Casares, Bao Sheng Loe, John Burden, Seán Ó hÉigeartaigh, José Hernández-Orallo:
How General-Purpose Is a Language Model? Usefulness and Safety with Human Prompters in the Wild. AAAI 2022: 5295-5303 - [c89]Fernando Martínez-Plumed, David Castellano Falcón, Carlos Monserrat Aranda, José Hernández-Orallo:
When AI Difficulty Is Easy: The Explanatory Power of Predicting IRT Difficulty. AAAI 2022: 7719-7727 - [c88]José Hernández-Orallo, Wout Schellaert, Fernando Martínez-Plumed:
Training on the Test Set: Mapping the System-Problem Space in AI. AAAI 2022: 12256-12261 - [c87]Ryan Burnell, John Burden, Danaja Rutar, Konstantinos Voudouris, Lucy Cheke, José Hernández-Orallo:
Not a Number: Identifying Instance Features for Capability-Oriented Evaluation. IJCAI 2022: 2827-2835 - [c86]Anthony G. Cohn, José Hernández-Orallo, Julius Sechang Mboli, Yael Moros-Daval, Zhiliang Xiang, Lexin Zhou:
A Framework for Categorising AI Evaluation Instruments. EBeM@IJCAI 2022 - [c85]Cèsar Ferri, José Hernández-Orallo, Jan Arne Telle:
Non-Cheating Teaching Revisited: A New Probabilistic Machine Teaching Model. IJCAI 2022: 2973-2979 - [c84]Songül Tolan, Annarosa Pesole, Fernando Martínez-Plumed, Enrique Fernández-Macías, José Hernández-Orallo, Emilia Gómez:
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks (Extended Abstract). IJCAI 2022: 5777-5781 - [c83]Konstantinos Voudouris, Niall Donnelly, Danaja Rutar, Ryan Burnell, John Burden, José Hernández-Orallo, Lucy Cheke:
Evaluating Object Permanence in Embodied Agents using the Animal-AI Environment. EBeM@IJCAI 2022 - [c82]Lexin Zhou, Fernando Martínez-Plumed, José Hernández-Orallo, Cèsar Ferri, Wout Schellaert:
Reject Before You Run: Small Assessors Anticipate Big Language Models. EBeM@IJCAI 2022 - [c81]Yue Zhao, José Hernández-Orallo:
Heterogeneity Breaks the Game: Evaluating Cooperation-Competition with Multisets of Agents. ECML/PKDD (4) 2022: 167-182 - [p1]José Hernández-Orallo, Cèsar Ferri:
Teaching and Explanation: Aligning Priors between Machines and Humans. Human-Like Machine Intelligence 2022: 171-196 - [e10]Gabriel Pedroza, José Hernández-Orallo, Xin Cynthia Chen, Xiaowei Huang, Huáscar Espinoza, Mauricio Castillo-Effen, John A. McDermid, Richard Mallah, Seán Ó hÉigeartaigh:
Proceedings of the Workshop on Artificial Intelligence Safety 2022 (SafeAI 2022) co-located with the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI2022), Virtual, February, 2022. CEUR Workshop Proceedings 3087, CEUR-WS.org 2022 [contents] - [e9]José Hernández-Orallo, Lucy Cheke, Joshua B. Tenebaum, Tomer D. Ullman, Fernando Martínez-Plumed, Danaja Rutar, John Burden, Ryan Burnell, Wout Schellaert:
Proceedings of the Workshop on AI Evaluation Beyond Metrics co-located with the 31st International Joint Conference on Artificial Intelligence (IJCAI-ECAI 2022), Vienna, Austria, July 25th, 2022. CEUR Workshop Proceedings 3169, CEUR-WS.org 2022 [contents] - [e8]Gabriel Pedroza, Xin Cynthia Chen, José Hernández-Orallo, Xiaowei Huang, Huáscar Espinoza, Richard Mallah, John A. McDermid, Mauricio Castillo-Effen:
Proceedings of the Workshop on Artificial Intelligence Safety 2022 (AISafety 2022) co-located with the Thirty-First International Joint Conference on Artificial Intelligence and the Twenty-Fifth European Conference on Artificial Intelligence (IJCAI-ECAI-2022), Vienna, Austria, July 24-25, 2022. CEUR Workshop Proceedings 3215, CEUR-WS.org 2022 [contents] - [i34]Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R. Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, Agnieszka Kluska, Aitor Lewkowycz, Akshat Agarwal, Alethea Power, Alex Ray, Alex Warstadt, Alexander W. Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Ambrose Slone, Ameet Rahane, Anantharaman S. Iyer, Anders Andreassen, Andrea Madotto, Andrea Santilli, Andreas Stuhlmüller, Andrew M. Dai, Andrew La, Andrew K. Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassum, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabharwal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakas, B. Ryan Roberts, Bao Sheng Loe, Barret Zoph, Bartlomiej Bojanowski, Batuhan Özyurt, Behnam Hedayatnia, Behnam Neyshabur, Benjamin Inden, Benno Stein, Berk Ekmekci, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, Cèsar Ferri Ramírez, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Chris Waites, Christian Voigt, Christopher D. Manning, Christopher Potts, Cindy Ramirez, Clara E. Rivera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, Daniel Freeman, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yuret, Derek Chen, Derek Tam, Dieuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodolà, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A. Chi, Ethan Dyer, Ethan J. Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fatemeh Siar, Fernando Martínez-Plumed, Francesca Happé, François Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Wang, Gonzalo Jaimovitch-López, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Hannah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Shevlin, Hinrich Schütze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B. Simon, James Koppel, James Zheng, James Zou, Jan Kocon, Jana Thompson, Janelle Wingfield, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse H. Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U. Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, José Hernández-Orallo, Joseph Boudeman, Joseph Guerr, Joseph Jones, Joshua B. Tenenbaum, Joshua S. Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh D. Dhole, Kevin Gimpel, Kevin Omondi, Kory Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Kumar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras Ochando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Ludwig Schmidt, Luheng He, Luis Oliveros Colón, Luke Metz, Lütfi Kerem Senel, Maarten Bosma, Maarten Sap, Maartje ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, María José Ramírez-Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L. Leavitt, Matthias Hagen, Mátyás Schubert, Medina Baitemirova, Melody Arnaud, Melvin McElrath, Michael A. Yee, Michael Cohen, Michael Gu, Michael I. Ivanitskiy, Michael Starritt, Michael Strube, Michal Swedrowski, Michele Bevilacqua, Michihiro Yasunaga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Aminnaseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T., Nanyun Peng, Nathan A. Chi, Nayeon Lee, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas Muennighoff, Nitish Shirish Keskar, Niveditha Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Elbaghdadi, Omer Levy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormolabashi, Peiyuan Liao, Percy Liang, Peter Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Milkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A. Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan LeBras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Ruslan Salakhutdinov, Ryan Chi, Ryan Lee, Ryan Stovall, Ryan Teehan, Rylan Yang, Sahib Singh, Saif M. Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R. Bowman, Samuel S. Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A. Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima (Shammie) Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehaene, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanie Lin, Stephen Prasad, Steven T. Piantadosi, Stuart M. Shieber, Summer Misherghi, Svetlana Kiritchenko, Swaroop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsu Hashimoto, Te-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timofei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay V. Ramasesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Seid, Zhuoye Zhao, Zijian Wang, Zijie J. Wang, Zirui Wang, Ziyi Wu:
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models. CoRR abs/2206.04615 (2022) - 2021
- [j50]Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, Marek J. Sergot:
Making sense of sensory input. Artif. Intell. 293: 103438 (2021) - [j49]Amparo Alonso-Betanzos, Pedro Cabalar, Graçaliz Pereira Dimuro, Marcos García, José Hernández-Orallo, Raquel Hervás, Angeles Manjarés, Fernando Martínez-Plumed, Inmaculada Mora-Jiménez, Miquel Sànchez-Marrè:
Editor's Note. Int. J. Interact. Multim. Artif. Intell. 6(5): 4-7 (2021) - [j48]Fernando Martínez-Plumed, Cèsar Ferri, David Nieves, José Hernández-Orallo:
Missing the missing values: The ugly duckling of fairness in machine learning. Int. J. Intell. Syst. 36(7): 3217-3258 (2021) - [j47]Songül Tolan, Annarosa Pesole, Fernando Martínez-Plumed, Enrique Fernández-Macías, José Hernández-Orallo, Emilia Gómez:
Measuring the Occupational Impact of AI: Tasks, Cognitive Abilities and AI Benchmarks. J. Artif. Intell. Res. 71: 191-236 (2021) - [j46]Lidia Contreras Ochando, Cèsar Ferri, José Hernández-Orallo:
AUTOMAT[R]IX: learning simple matrix pipelines. Mach. Learn. 110(4): 779-799 (2021) - [j45]Fernando Martínez-Plumed, Emilia Gómez, José Hernández-Orallo:
Futures of artificial intelligence through technology readiness levels. Telematics Informatics 58: 101525 (2021) - [j44]Fernando Martínez-Plumed, Lidia Contreras Ochando, Cèsar Ferri, José Hernández-Orallo, Meelis Kull, Nicolas Lachiche, María José Ramírez-Quintana, Peter A. Flach:
CRISP-DM Twenty Years Later: From Data Mining Processes to Data Science Trajectories. IEEE Trans. Knowl. Data Eng. 33(8): 3048-3061 (2021) - [c80]John Burden, José Hernández-Orallo, Seán Ó hÉigeartaigh:
Negative Side Effects and AI Agent Indicators: Experiments in SafeLife. SafeAI@AAAI 2021 - [c79]Fernando Martínez-Plumed, José Hernández-Orallo:
Project-Based Learning for Scaffolding Data Scientists' Skills. ICCSE 2021: 758-763 - [c78]Gust Verbruggen, Lidia Contreras Ochando, Cèsar Ferri, José Hernández-Orallo, Luc De Raedt:
Muppets: Multipurpose Table Segmentation. IDA 2021: 389-401 - [c77]Gonzalo Jaimovitch-López, David Castellano Falcón, César Ferri, José Hernández-Orallo:
Think Big, Teach Small: Do Language Models Distil Occam's Razor? NeurIPS 2021: 1610-1623 - [c76]Manuel Garcia-Piqueras, José Hernández-Orallo:
Optimal Teaching Curricula with Compositional Simplicity Priors. ECML/PKDD (1) 2021: 705-721 - [e7]Huáscar Espinoza, John A. McDermid, Xiaowei Huang, Mauricio Castillo-Effen, Xin Cynthia Chen, José Hernández-Orallo, Seán Ó hÉigeartaigh, Richard Mallah:
Proceedings of the Workshop on Artificial Intelligence Safety 2021 (SafeAI 2021) co-located with the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021), Virtual, February 8, 2021. CEUR Workshop Proceedings 2808, CEUR-WS.org 2021 [contents] - [e6]Huáscar Espinoza, John A. McDermid, Xiaowei Huang, Mauricio Castillo-Effen, Xin Cynthia Chen, José Hernández-Orallo, Seán Ó hÉigeartaigh, Richard Mallah, Gabriel Pedroza:
Proceedings of the Workshop on Artificial Intelligence Safety 2021 co-located with the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI 2021), Virtual, August, 2021. CEUR Workshop Proceedings 2916, CEUR-WS.org 2021 [contents] - [i33]Tijl De Bie, Luc De Raedt, José Hernández-Orallo, Holger H. Hoos, Padhraic Smyth, Christopher K. I. Williams:
Automating Data Science: Prospects and Challenges. CoRR abs/2105.05699 (2021) - [i32]Manuel Garcia-Piqueras, José Hernández-Orallo:
Conditional Teaching Size. CoRR abs/2107.07038 (2021) - [i31]Radosvet Desislavov, Fernando Martínez-Plumed, José Hernández-Orallo:
Compute and Energy Consumption Trends in Deep Learning Inference. CoRR abs/2109.05472 (2021) - 2020
- [j43]Grace Bang, Guy Barash, Ryan Beal, Jacques Calì, Mauricio Castillo-Effen, Xin Cynthia Chen, Niyati Chhaya, Rachel Cummings, Rohan Dhoopar, Sebastijan Dumancic, Huáscar Espinoza, Eitan Farchi, Ferdinando Fioretto, Raquel Fuentetaja, Christopher William Geib, Odd Erik Gundersen, José Hernández-Orallo, Xiaowei Huang, Kokil Jaidka, Sarah Keren, Seokhwan Kim, Michel Galley, Xiaomo Liu, Tyler Lu, Zhiqiang Ma, Richard Mallah, John A. McDermid, Martin Michalowski, Reuth Mirsky, Seán Ó hÉigeartaigh, Deepak Ramachandran, Javier Segovia-Aguas, Onn Shehory, Arash Shaban-Nejad, Vered Shwartz, Siddharth Srivastava, Kartik Talamadupula, Jian Tang, Pascal Van Hentenryck, Dell Zhang, Jian Zhang:
The Association for the Advancement of Artificial Intelligence 2020 Workshop Program. AI Mag. 41(4): 100-114 (2020) - [j42]David Nieves, María José Ramírez-Quintana, Carlos Monserrat Aranda, Cèsar Ferri, José Hernández-Orallo:
Learning alternative ways of performing a task. Expert Syst. Appl. 148: 113263 (2020) - [j41]José Hernández-Orallo:
Twenty Years Beyond the Turing Test: Moving Beyond the Human Judges Too. Minds Mach. 30(4): 533-562 (2020) - [j40]Fernando Martínez-Plumed, José Hernández-Orallo:
Dual Indicators to Analyze AI Benchmarks: Difficulty, Discrimination, Ability, and Generality. IEEE Trans. Games 12(2): 121-131 (2020) - [c75]John Burden, José Hernández-Orallo:
Exploring AI Safety in Degrees: Generality, Capability and Control. SafeAI@AAAI 2020: 36-40 - [c74]Fernando Martínez-Plumed, Songül Tolan, Annarosa Pesole, José Hernández-Orallo, Enrique Fernández-Macías, Emilia Gómez:
Does AI Qualify for the Job?: A Bidirectional Model Mapping Labour and AI Intensities. AIES 2020: 94-100 - [c73]José Hernández-Orallo:
AI Safety Landscape From short-term specific system engineering to long-term artificial general intelligence. DSN Workshops 2020: 72-73 - [c72]Raül Fabra-Boluda, Cèsar Ferri, Fernando Martínez-Plumed, José Hernández-Orallo, M. José Ramírez-Quintana:
Family and Prejudice: A Behavioural Taxonomy of Machine Learning Techniques. ECAI 2020: 1135-1142 - [c71]José Hernández-Orallo, Jan Arne Telle:
Finite and Confident Teaching in Expectation: Sampling from Infinite Concept Classes. ECAI 2020: 1182-1189 - [c70]José Hernández-Orallo, Fernando Martínez-Plumed, Shahar Avin, Jess Whittlestone, Seán Ó hÉigeartaigh:
AI Paradigms and AI Safety: Mapping Artefacts and Techniques to Safety Issues. ECAI 2020: 2521-2528 - [c69]Fernando Martínez-Plumed, José Hernández-Orallo, Emilia Gómez:
Tracking AI: The Capability Is (Not) Near. ECAI 2020: 2915-2916 - [e5]Huáscar Espinoza, José Hernández-Orallo, Xin Cynthia Chen, Seán S. ÓhÉigeartaigh, Xiaowei Huang, Mauricio Castillo-Effen, Richard Mallah, John A. McDermid:
Proceedings of the Workshop on Artificial Intelligence Safety, co-located with 34th AAAI Conference on Artificial Intelligence, SafeAI@AAAI 2020, New York City, NY, USA, February 7, 2020. CEUR Workshop Proceedings 2560, CEUR-WS.org 2020 [contents] - [e4]Huáscar Espinoza, John A. McDermid, Xiaowei Huang, Mauricio Castillo-Effen, Xin Cynthia Chen, José Hernández-Orallo, Seán Ó hÉigeartaigh, Richard Mallah:
Proceedings of the Workshop on Artificial Intelligence Safety 2020 co-located with the 29th International Joint Conference on Artificial Intelligence and the 17th Pacific Rim International Conference on Artificial Intelligence (IJCAI-PRICAI 2020), Yokohama, Japan, January, 2021. CEUR Workshop Proceedings 2640, CEUR-WS.org 2020 [contents] - [i30]Richard Evans, José Hernández-Orallo, Johannes Welbl, Pushmeet Kohli, Marek J. Sergot:
Evaluating the Apperception Engine. CoRR abs/2007.05367 (2020)
2010 – 2019
- 2019
- [j39]Fernando Martínez-Plumed, Ricardo B. C. Prudêncio, Adolfo Martínez Usó, José Hernández-Orallo:
Item response theory in AI: Analysing machine learning classifiers at the instance level. Artif. Intell. 271: 18-42 (2019) - [j38]Guy Barash, Mauricio Castillo-Effen, Niyati Chhaya, Peter Clark, Huáscar Espinoza, Eitan Farchi, Christopher W. Geib, Odd Erik Gundersen, Seán Ó hÉigeartaigh, José Hernández-Orallo, Chiori Hori, Xiaowei Huang, Kokil Jaidka, Pavan Kapanipathi, Sarah Keren, Seokhwan Kim, Marc Lanctot, Danny Lange, Julian J. McAuley, David R. Martinez, Marwan Mattar, Mausam, Martin Michalowski, Reuth Mirsky, Roozbeh Mottaghi, Joseph C. Osborn, Julien Pérolat, Martin Schmid, Arash Shaban-Nejad, Onn Shehory, Biplav Srivastava, William W. Streilein, Kartik Talamadupula, Julian Togelius, Koichiro Yoshino, Quanshi Zhang, Imed Zitouni:
Reports of the Workshops Held at the 2019 AAAI Conference on Artificial Intelligence. AI Mag. 40(3): 67-78 (2019) - [j37]Cèsar Ferri, José Hernández-Orallo, Peter A. Flach:
Setting decision thresholds when operating conditions are uncertain. Data Min. Knowl. Discov. 33(4): 805-847 (2019) - [j36]José Hernández-Orallo:
AI Generality and Spearman's Law of Diminishing Returns. J. Artif. Intell. Res. 64: 529-562 (2019) - [j35]Jan Arne Telle, José Hernández-Orallo, Cèsar Ferri:
The teaching size: computable teachers and learners for universal languages. Mach. Learn. 108(8-9): 1653-1675 (2019) - [j34]José Hernández-Orallo:
Gazing into Clever Hans machines. Nat. Mach. Intell. 1(4): 172-173 (2019) - [c68]José Hernández-Orallo, Fernando Martínez-Plumed, Shahar Avin, Seán Ó hÉigeartaigh:
Surveying Safety-relevant AI Characteristics. SafeAI@AAAI 2019 - [c67]José Hernández-Orallo, Karina Vold:
AI Extenders: The Ethical and Societal Implications of Humans Cognitively Extended by AI. AIES 2019: 507-513 - [c66]Matthew Crosby, Benjamin Beyret, Murray Shanahan, José Hernández-Orallo, Lucy Cheke, Marta Halina:
The Animal-AI Testbed and Competition. NeurIPS (Competition and Demos) 2019: 164-176 - [c65]Lidia Contreras Ochando, Cèsar Ferri, José Hernández-Orallo:
Automating Common Data Science Matrix Transformations. PKDD/ECML Workshops (1) 2019: 17-27