Ten university teams selected for the Alexa Prize TaskBot Challenge 2

Amazon today announced that ten teams from around the globe have been selected to participate in the Alexa Prize TaskBot Challenge Year 2, a university challenge focused on developing multimodal (voice and vision) conversational agents that help customers complete tasks that require multiple steps and decisions.

The Alexa Prize is a flagship industry-academia collaboration dedicated to accelerating the science of conversational artificial intelligence (AI) and multimodal human-AI interactions.

“Prize competitions provide an agile science experimentation framework for researchers and students, encouraging them to explore transformational ideas at the edge of what is achievable,” said Reza Ghanadan, senior principal scientist at Alexa AI and head of the Alexa Prize. “We have developed the CoBot platform and tools to lower the barriers to AI innovation for both the academic research community and students interested in conversational AI assistants. These tools allow students to quickly implement their solutions at scale in the real world with Alexa, and then observe, evaluate and improve their research results using feedback from Alexa customers.”

The teams selected for the challenge, which began in January, have five returning participants – including the top three in the most recent challenge – and five new universities.

Prizes for overall performance in the competition will be $500,000 for first place, $100,000 for second place and $50,000 for third place. These prizes will be awarded to the students in the teams with the best overall performance.

“I’m excited to see new teams join the second year of the competition, along with returning teams who, by competing again, signal to us that they found value in the TaskBot challenge,” said Yoelle Maarek, vice president of research and science for Amazon Shopping.

“We expect these talented graduate students to continue to surprise us, as well as Amazon customers, this year. Connecting academia, Amazonians, and actual customers experimenting with taskbots is a winning combination to keep pushing the boundaries of science in conversational AI for Alexa to delight and ease the lives of millions of customers.”

The Alexa Prize is a competition for college students dedicated to advancing the field of conversational AI. Launched in 2016, the program was created to recognize students from around the world who are changing the way we interact with technology.

TaskBot Challenge 2 teams are working to solve one of the toughest problems in conversational AI – creating next-generation conversational AI experiences that delight customers by meeting their evolving needs as they perform complex tasks. This challenge builds on the foundation of the Alexa Prize to provide universities with a unique opportunity to test cutting-edge machine learning models with actual customers at scale.

The Alexa Prize TaskBot challenge provides a realistic scenario of real-user multimodal interactions, making this the perfect setting to observe and measure human-bot conversations and AI algorithms in a cutting-edge setting.

Rafael Ferreira, NOVA School of Science and Technology, Team TWIZ

Our vision for EvoquerBOT combines improving task completion rates and increasing user satisfaction. To this end, we provide innovative solutions to fundamental NLP challenges.

Haoran Zhang, Penn State University, Team EvoquerBOT

We are particularly interested in developing innovative ways to successfully coordinate multiple modalities, such as visual and verbal elements, and create a more engaging and intuitive user experience.

Lingbo Mo, Ohio State University, Team Taco 2.0

The GRILL team is excited to continue bringing cutting-edge AI research to improve people’s lives. Our research team is working on new possibilities for foundational models that understand text, images and the surrounding world.

Sophie Fischer, University of Glasgow, Team GRILL

The competition allows us to create interfaces with the general public in a production environment – ​​it is a unique opportunity to connect our research with our career goals.

Baber Khalid, Rutgers University, Team DiWBot

We are very excited to be a part of the community and look forward to working with the Alexa team and other teams.

Anthony Sicilia, University of Pittsburgh, Team ISABEL

The Alexa Prize TaskBot Challenge combines a wide range of tasks across multiple domains with multimodal output. This is the ultimate test for any moonshot concept, and we can’t wait to see what the real world has in store for us.

Rey (Alex) Gonzalez, Purdue University, Team BoilerBot

Entering this competition is an incredible opportunity that will allow us to conduct applied research and send it to real users.

Chris Samarinas, University of Massachusetts Amherst, Team Maruna

Although artificial intelligence has experienced explosive development in the past decade, there is still a gap between research and real-world application. The TaskBot Challenge gives us a unique opportunity to explore multimodal AI in practical situations.

Kaizhi Zheng University of California, Santa Cruz-Amherst, Team Sage

Our bot will make adaptive conversation a reality by allowing customers to follow personal decisions through the execution of multiple, sequential sub-tasks and adapt to the tools, materials or ingredients available to the user by suggesting suitable substitutes and alternatives.

Afrina Tabassum

TaskBot is the first conversational AI challenge to incorporate multimodal customer experiences, so in addition to receiving verbal instructions, customers with Echo Show or Fire TV devices can also be presented with step-by-step instructions, images or diagrams that enhance task guidance.

This year’s challenge has been expanded to include more hobbies and activities around the home. Participating teams were asked to suggest interesting ways to incorporate visual aids into each turn of conversation when a screen is available. Innovative ideas for improving the presentation of visual aids as well as coordinating visual and verbal modalities were part of the team selection criteria.

Each university selected for the challenge receives a $250,000 research grant, Alexa-enabled devices, free Amazon Web Services (AWS) cloud computing services to support their research and development efforts, access to Amazon researchers, the CoBot (conversational bot) toolkit, and other tools such as automated speech recognition through Alexa, dataset generation, and conversational design, neural detection, neural detection, neural detection, Alexa Prize team.

“Alexa, let’s work together”

The university teams’ taskbots will be available for Alexa customers to engage with in May 2023, with a finals event held in September and winners announced later that month.

As with the previous challenge, Alexa customers can interact with Teams’ taskbots when they become available in May by saying, “Alexa, let’s work together.” Until then, “Alexa, let’s work together” will lead you to conversations with the past 2022 challenge winners and the Alexa Prize TaskBot.

After beginning the interaction, Alexa customers then receive a short message informing them that they are interacting with an Alexa Prize university taskbot, before being randomly paired with one of the participating taskbots.

After completing the conversation with the taskbot, which customers can do at any time, the customer is prompted for a verbal rating, followed by an opportunity to provide additional feedback. The interactions, ratings and feedback are shared with the teams to help them improve their taskbots. Customer ratings are also used to determine which college teams advance to the semi-finals and finals.

Our goal is to contribute to the multimodal conversational AI field and move it closer to the way humans perceive, reason, and communicate through multimodal information.

João Magalhães, Associate Professor, NOVA School of Science and Technology, Team TWIZ

We look forward to the challenge because it is the perfect platform for creating multimodal, task-oriented dialogue systems that elevate the user experience and engagement.

Rui Zhang, Assistant Professor, Penn State University, Team EvoquerBOT

Through this TaskBot challenge, we hope our work can expand the horizons of conversational AI along dimensions such as dialogue depth, multimodal coordination, common sense, and learning of use.

Huan Sun, Associate Professor, Ohio State University, Team Taco 2.0

The GRILL team is creating the next generation of open assistants that understand and use knowledge about the world and can communicate effectively to inform and educate.

Jeff Dalton, Associate Professor, University of Glasgow, Team GRILL

Our TaskBot will help people get things done through personalized, adaptive and context-aware conversational interaction by combining our research findings with the advanced features of Alexa devices.

Matthew Stone, Professor, Rutgers University, Team DiWBot

We are working to make conversational AI technology more inclusive and collaborative. Including Alexa can collaborate with users from different cultures and with different communication options and preferences.

Malihe Alikhani, Assistant Professor, University of Pittsburgh, Team ISABEL

We hope to develop a task-oriented system that can interact with users based on their knowledge level, experience, and communication preference.

Julia Rayz, Professor, Purdue University, Team BoilerBot

Success in the previous TaskBot Challenge required teams to tackle many difficult AI obstacles. The challenge required the fusion of several AI techniques, including knowledge representation and inference, common sense and causal reasoning, and language understanding and generation.

The “GRILLBot” team from the University of Glasgow won the TaskBot 1 challenge and earned a $500,000 prize for its achievement. Teams from NOVA School of Science and Technology (Portgual) and Ohio State University won second and third place, respectively.

Research papers from the Amazon Alexa Prize team and each of the competing teams can be viewed and downloaded here.