Millions of customers around the world rely on Amazon’s huge online catalog, which contains information about the HUGS of millions of products to make informed purchase decisions. To ensure that catalog data is understanding, consists and accurate, the Amazon catalog team uses a wide range of machine learning models-inclusive generative-AA models that synthesize textual and visual information from seller lists, producer sites, customer reviews and other sources of enriching catalog.
To ensure that the enriched data improves the customer experience, Amazon Catalog A/B completes the experiment for our customers’ enriched information, while others encourage the current alternative.
But A/B testing can Curry -Opportunity Costs as we delay the roll -out of catalog improvements for some of our customers, and maintenance -To Backend systems are resource -intensive. To tackle these challenges, we offer two different scientific approaches.
Machine-learning-based extrapolation model
First we seek to drive as if possible. We have developed a scalable machine learning -based extrapolate model that effectively incorporated insights gained from prior experience with enrichment initiatives and used them for new contexts. We tailor the causal-mandom-forest approach, which in itself is an extension of the classic random forest algorithm, for our setting.
During training on existing A/B experiences, the algorithm randomly chooses training and validation data sets and generates an ensemble of causal decision trees. Each tree divides the products involved in the experiment into smaller subgroups, sorted by similarity between functions that balance it within the sample-care of the observed results in treatment status and the out-of-testing performance on the validation data set. We then gather the different predictions from all causal trees to generate a prediction of the effect of enrichment that gets different product functions. After training the model, we can validate it in further experiment to compare predicted with new treatment effects.
The validated model lets us test, but there are different in the answers to an enrichment across products. If so, we can focus our enrichment efforts on the product groups that respond particularly well. In addition, we can use the estimates to predict and document the impact of our planned enrichments on our customers as we tackle different groups throughout the year. As an example, we can use the model to assess the impact of our efforts to correct and implement product information across the catalog with only a limited set of experience.
Bayesian Structural Time Series
When the performance of A/B experiment is not possible, we can contact observal modeling techniques, such as Bayesian-structural modeling modeling. This approach synthesizes ideas from time series analysis, synthetic control method and Bayesian statistics.
When we monitor the sale of all our products over time, we can pay any product group with a synthetic twin that reflects its sales benefit that accounts for sales trends and seasonal determination. If we enrich a group of products and observe a noticeable other in the sales benefit, we can attribute the change to our enrichment effort. Operating within the Bayese framework enables us to integrate prior knowledge from our various analyzes, included in A/B experience and effectively communicate uncertainty to our business stakeholders.
We have validated our observation model against our A/B experiment for selected usage boxes, where both methods are possible and have used this approval to evaluate the impact of our large machine learning systems that automatically classify products to improve their discovery through search or browsing.
With enriched data catalog, Amazon customers are able to make better informed and more convinced shopping decisions.
Recognitions: Philipp Eisenhauer