The first draft of this blog post was generated by Amazon Nova ProBased on detailed instructions from Amazon Science editors and several examples of previous submissions.
In a paper we present at the 2025 conference on computer vision and pattern recognition (CVPR), we introduce a new approach to image segmentation that scrosal different data and tasks. Traditional segmentation models, while being effective on isolated tasks, often streams as the number of new tasks or unknown scenarios is growing. Our proposed method, which uses a model we call a mixed-compliant transform (MQ-ENVER), aims to enable joint training and evaluation across multiple tasks and data sets.
Scalable segmentation
Image segmentation is a computer vision task that involves the breakdown of an image into different regions or segments. Each segment corresponds to another object or part of the stage. There are several types of segmentation tasks, including foreground/background segmentation (distinguishes objects at different distances), semantic segmentation (labeling each pixel as amounting to a particular object class) and occurrence segmentation (identify each pixel as associated occurrence of an object class).
“Scalabibility” means that a segmentation model can effectively be improvised with an increase in the size of its training data set, in the diversity of the tasks it performs, or condition. Most previous studies have focused on one or the other – data or task diversity. We both address at once.
A tale of two queries
In our paper, we show that a problem that prevents effective scalibility in segmentation models is designed to Object requests. An object request is a way of representing a hypothesis about objects in a scene – a hypothesis that can be tested against images.
There are two hand types of object requests. The first to which we red to as “taught inquiries” are learned vectors that interact with image functions and encoding rental and object class. Learned queries tend to work well at semantic segmentation as they do not contain object safety prices.
The second type of object request that we red to as one Conditional inquiryis Akin to two-step object detection: Regional suggestions are generated by a transformer codes, and then fed suggestions with high confidence in the transform cod as quies to generate the final prediction. Conditional queries are closely line with the object classes and are distinguished by object detection and occurrence segmentation on semantically well -defined objects.
Our approval is to combine stand types of queries, which improves the model’s ability to transfer across tasks. Our MQ-Past Model represents input using both lively inquiries and condition questions, and each layer of the decoder has a cross-screen mechanism, so the processing of the tilable quaries can factor in information from the conditional Kvquery treatment, and vice versa.
Synthetic data delivery
Mixed queries help with scalibility across segmentation tests, but the second aspect of scale in segmentation models is data set size. One of the most important challenges in the upscaling of segmentation models is the scarcity of high quality, annotated data. To make this limitation we offer to utilize synthetic data.
While segmentation data is scarce, object recognition data is plenty. Object recognition data set typically includes Boundary boxesBut rectangles that identify the image regions where labeled objects can be found.
Asking a trained segmentation model to segment only the object in a bounding box improves Nordic performance; We are thus able to use weaker segmentation models to convert object recognition data sets to segmentation data sets that may be to train stronger segmentation models.
Delimitation boxes can also focus automatic caption models on regions of interest in an image to provide the type of object classifications needed to train semantic segmentation and occurrence segmentation models.
Experimental results
We evaluated our approach using 15 data sets covering a number of segmentation tasks and found that with MQ-recently scaling of both the amount of training data and the diversity of tasks, the constant model’s segmentation functions are constantly stated.
E.g. On Seginw-Benchmark, which includes 25 data sets used for open vocabulary in-up segmentation evaluation, scaling of data and tasks from 100,000 samples to 600,000 increased performance 16%, measured by the ease of objection. Incorporation of synthetic data improved the benefit by another 14%and established a new technique.