Amazon announced today that a team from the Nova School of Science and Technology (FCT NOVA) in Portugal has achieved first place in Alexa Prize Taskbot Challenge 2. Complete complex tasks.
Taskbot is the first conversational AI challenge to incorporate multimodal customer experiences. During the competition, in addition to verbal instructions, some customers with Echo Show or four tvs were presented with step-by-sep instructions, images or charts to improve the task guidance.
“The most encouraging and impressive progress we, in the use of large language models on dialogue management,” said Michael Johnston, a used science director at Alexa AI, who leads the science and engineering teams that support the Alexa award. “Instead of just using LLMs to create candes resorts, teams explored to have an instructional following LLM drive throughout the conversation. I think cracking this problem for the task aid domain was the biggest contributing factor in the quality and naturalness we saw in the top executing bots.
Team Twiz, advised by João Magalhães, took $ 500,000 home to earn first place in the overall performance.
“I am very fond of the team’s creativity in designing the groundbreaking Twiz LLM,” Magalhães said. “Conversations about video content take CX to a whole new level, and I’m very proud to help pioneer video dialogue in the Alexa award. I think there’s a lot to explore here.”
This year’s challenge was expanded to include more hobbies and home activities. Teams were asked to find interesting ways to integrate visual aids into each conversation -when a screen is available. Innovative ideas for improving the presentation of visual aids as well as the coordination of visual and verbal states were part of the assessment criteria.
“User dialogues in Alexa Taskbot are unique and throw a new light into the performance of manual tasks,” said Rafael Ferreira, the Twiz team. “Living by these dialogues we learned that the use of Twiz enabled us to manage conversations in a more contextual and insightful way.”
Team Grill from the University of Glasgow, advised by Jeff Dalton, earned $ 100,000 for second place, and Team Isabel from the University of Pittsburgh, advised by Malihe Alikhani, earned $ 50,000 in the third place award. Working with the top three teams with the other participants is now caught in a number of research articles.
“Working on the Taskbot 2 challenge gave us the unique opportunity to develop and implement advanced language models,” said Sophie Fischer, Grill Team Lead. “We learned that it’s not just about model size or improved training, but about using models in new and creative ways to help people.”
Five university teams were chosen to participate in the last live interaction phase of the Taskbot Challenge 2 earlier this year. The teams were chosen based on, among other criteria, customer feedback and scientific profits for the technical papers produced by each team. The other two finalist teams were Team Plan-Bot from Virginia Tech, advised by Ismini Lourentzou; And Team Sage, advised by Xin (Eric) Wang, from the University of California, Santa Cruz.
“Compared to previous challenges, it was interesting to see how broadly generative AI and large language models are used,” Johnston said. “Previous challenges have used previous language models to generate Candesate’s response, but with the increase in linguistic models with great capacity with the ability to follow the instructions, teams use them for many different tasks needed to improve their bots.
“This included tasks such as classification, formulation of search queries, creation of synthetic data sets, creation of complete descriptions of tasks and more,” he continued. “Hold also explored different uses to make users easier to clarify and iterate on their input using the screen, and they also began to add awkward technology capabilities to increase the range of tasks to underestimated societies.”
Alexa customers interacted with the university tasks of Amazon Echo or four TV devices. Customer assessments and feedback helped the students teams to improve their bots when they joined.
Each university selected for the challenge received a $ 250,000 research fellow, Alexa-enabled devices, Free Amazon Web Services (AWS) Cloud Computing Services to support their research and development efforts, access to Amazon scientists, COBOT (CONVERSATIONAL BOT) Tool sets and other tools such as Automet Specech through Alexa, Neural Detection Response generation models, conversational data and design guidance and development support the Alexa Prize.
During the competition, customers engaged with the university team’s assignment bots. After starting the interaction, customers drew a short message informing them that they interacted with an Alexa Prize University Taskbot before they were randomly connected to one of the participating Taskbots.
After the conversation with Taskbot, the customer was quick about a verbal rating, followed by an opportunity to provide additional feedback. The interactions, ratings and feedback we shared with the teams to help them improve their task bots. Customer assessments were also used to decide which university teams went on to the semi -finals and the final.
Success in the former Taskbot Challenge required teams to tackle many difficult AIT bars. The challenge required the merger of several AI techniques, including knowledge representation and conclusions, commonsense and causal relationships and language understanding and generation.
“The performance of some of the Taskbots in the second year of the competition improved drastically compared to Taskbot 1,” said Eugene Agichtein, a computer science professor at Emory University and Amazon Scholar, who also served as faculty advisor for two of Emory’s Alexa teams. “I was reviewed to see Spring Forward because of the learned and data and models created in the first year of the Taskbot competition, combined with improvises in LLM technology.
The “Grillbot” team from the University of Glasgow won the Taskbot 1 Challenge in 2022 and earned a price of $ 500,000 for its performance. Teams from the Nova School of Science and Technology (Portugal) and Ohio State University served at second and third plate awards, respective.