Why new product demand forecasting is challenging
Walk into any Target and head to the shampoo aisle. You’ll find at least 75 choices: new ingredients, unique scents, hair-type formulas, and everything in between.
Today’s consumers expect variety. Not just more choices, but new ones, driven by personal preferences, seasonal trends, and whatever’s popular on social media this week.
The result is a world of constant product proliferation. New items are launched fast, and many don’t stick around long. For businesses, this creates a high-stakes challenge: how to forecast demand for an ever-changing product portfolio.
Underforecasting demand leads to empty store shelves, lost opportunity for revenue and poor customer experience.
Overforecasting demand leads to cash tied up in excess inventory as products collect dust in storage.
How AI and unsupervised learning improve forecast accuracy
To most people, AI means ChatGPT, a conversational tool that can answer questions on nearly any topic. But in supply chain planning, that’s just one branch of a much larger tree.
AI has been used in supply chain for years, especially to tackle one of the toughest challenges: forecasting demand for new products.
Traditional forecasting relies on historical data, but with new product introductions, that data doesn’t exist yet. That’s where unsupervised learning comes in – a key branch of AI.
Think of it like this: if you’re launching a new sports drink, wouldn’t you want to know how similar drinks performed in their first few weeks?
Unsupervised learning groups new products with similar existing ones using machine learning algorithms. These clusters form around shared traits like price, brand, packaging, and expected behavior. By analyzing how similar products performed, we can estimate demand for a new product, even before it launches.
AI has been used in supply chain for years, especially to tackle one of the toughest challenges: forecasting demand for new products.
What features does AI use to cluster new products?
With unsupervised learning, the objective is to group (or cluster) similar products so we can derive patterns previously observed. The model is only as good as the data we feed it, and that starts with the features.
Features describe each product, such as brand, pack size, price point, target demographic, shelf placement, seasonality, and more. Ideally these come from internal systems; sometimes they require being created or curated based on context and domain knowledge.
Features of the new product are compared to other products in the portfolio. There are two ways to match features. The first is with manual feature engineering, where the user decides which features to use to find similar products. The second approach is automated feature selection, where the forecast engine identifies which features to use when identifying similarities. Each is a valid approach, with advantages and drawbacks.
Manual feature engineering gives the user more control and clarity. They decide what matters most:
- Will brand loyalty influence sales?
- Are there regional factors that affect adoption rates?
- Is this a price sensitive category?
This makes it easier to apply business logic and explain the model in planning meetings.
Automated feature selection, on the other hand, helps scale forecasts across a fast-moving portfolio. The model evaluates hundreds of variables and identifies the most predictive ones. This approach excels when:
- You’re launching many new SKUs over a short period of time
- You’re working with high dimensional data
- You want to uncover hidden demand drivers
From cluster analysis to real-time forecast adjustments
Once the product is placed in a cluster, that cluster’s previous product launches are evaluated.
- What was the rate of sales growth?
- How did the rate change throughout the launch period?
- How long did it take to reach peak demand?
These patterns form the basis for the initial forecast.
Once the product is launched, real-world data begins to flow in: shipments, POS data, channel inventory, social sentiment, and much more. The model begins to shift gears, transitioning from look-alike-based forecasting to real-time forecasting based on what’s actually happening with the product in the market.
The real world is messy – that’s why this is applied AI. There’s one universal truth in forecasting: your predictions will be wrong. The objective is to identify discrepancies early and make adjustments quickly, so the supply chain has more time to react. This is where e2open’s demand sensing technology can help you understand:
- Is the product selling faster or slower than anticipated?
- Is the new product stealing demand from old products?
- Is a promotion impacting the demand during launch?
- Are there supply issues impacting sales?
Automated, real-time adjustments paired with intelligent alerts for the user enable the business to stay on top of product launch progress.
Case study: How e2open’s AI technology boosted forecast accuracy
E2open founded the concept of demand sensing many years ago, which started under the name of Terra Technology. For years, using unsupervised learning to forecast demand for new products has been a critical capability for our customers to ensure an accurate forecast and ultimately a successful product launch.
Here’s an example – e2open partners with a major Consumer Packaged Goods (CPG) company to execute demand forecasting activities. This methodology significantly improved forecast performance. This particular company has new products which make up ~7% of the total product portfolio.
Prior to using e2open’s AI technology, planners were responsible for manually developing the new product forecast. They relied on their best judgment. The performance of the existing approach was clear – new product forecasts were significantly underperforming the other 93% of the mature product portfolio. The MAPE (Mean Absolute Percentage Error – a widely accepted forecast accuracy metric) for new products was significantly worse than mature products. The bias was 11% – indicating the forecast consistently overestimated true demand.
Using e2open’s demand sensing technology, AI was applied to help solve this challenge. The portfolio was scanned using automated feature engineering to identify attributes that indicate the expected launch pattern. These were then placed into clusters. When a new product was being launched, it would be matched to a cluster based on the attributes. Through scaling and pattern matching techniques, a forecast for the new product was generated.
The data pointed to a clear conclusion – e2open’s demand sensing technology provided significant improvement. MAPE improved 10-34% from the baseline. Forecast bias was reduced 32-59%.
The significant performance improvement is attributed to a couple key factors:
- The technology was able to identify specific features that correlated new products to products previously launched – something no planner could easily spot on their own.
- Using a data-driven approach removed the inherent bias built into a manual, consensus-driven process where people bring emotions, preferences and perspective shaped by past experiences.
The future of AI-driven demand forecasting
Forecasting new product demand with AI is a proven technique to increase forecast accuracy, and ultimately, business outcomes. Generative AI is getting a lot of attention right now, but let’s not forget that unsupervised learning continues to be an option many businesses can leverage to incorporate more AI into their supply chain strategy.
E2open was the first to commercialize demand sensing, and the technology has continued to evolve since then. Later this year, e2open will deliver its next generation of demand sensing, bringing a range of performance improvements and expanded capabilities.
Interested in improving your new product demand forecasting with AI? Contact us to learn how e2open’s demand sensing technology can optimize your supply chain planning and boost forecast accuracy.
