Traditional Methods vs AI Methods in Forecasting: When Simplicity Outperforms Complexity

Forecasting remains an indispensable tool for businesses to make informed decisions about future trends, demands, and opportunities. Accurate forecasts can lead to better resource allocation, inventory management, and strategic planning. When it comes to forecasting, data scientists have many tools at their disposal, ranging from traditional and time-tested methods to advanced artificial intelligence (AI) techniques. While AI methods have gained significant attention in recent years, traditional forecasting methods like Moving Average, Exponential Moving Average, and Linear Regression Forecasting still hold their ground in many real-world scenarios. As a data scientist with substantial experience in machine learning and big data analytics, I will shed light on why, in most cases, traditional forecasting methods remain preferable to their more advanced counterparts.

The Case for Traditional Forecasting Methods

1. Ease of Explanation

One of the most compelling reasons to favor traditional forecasting methods is their simplicity and ease of explanation. When presenting forecasts to non-technical stakeholders or decision-makers, it’s crucial that the methodology behind the predictions is understandable. Traditional methods are inherently transparent, making it easier to communicate how a forecast was generated. Furthermore, once implemented, documented, and explained, many users within the organization can operate and even use the forecasting tools to solve other problems or inefficiencies.

Moving Average

Moving Average (MA) is a fundamental method in time series forecasting. It calculates the average of a fixed window of past data points to predict future values. The formula is straightforward:

[ MA_t = \frac{X_{t-1} + X_{t-2} + \ldots + X_{t-n}}{n} ]

Here, (MA_t) represents the forecast for time (t), and (X_{t-1}, X_{t-2}, \ldots, X_{t-n}) are the past data points within the window of size (n).

Here is some pseudocode for how it would appear in an Microsoft Excel/Google Sheets:

=AVERAGE(A2:A6), =AVERAGE(A3:A7) and so on.

Exponential Moving Average

Exponential Moving Average (EMA) is a slight variation of the Moving Average method. It assigns exponentially decreasing weights to past data points, giving more importance to recent observations. The formula for EMA is as follows:

[ EMA_t = \alpha \cdot X_t + (1 - \alpha) \cdot EMA_{t-1} ]

Here, (EMA_t) is the forecast for time (t), (X_t) is the current observation, and (\alpha) is the smoothing factor.

Here is some pseudocode for how it would appear in an Microsoft Excel/Google Sheets:

EMV = [Latest Value - Previous EMA] * (2/n+1) + Previous EMA

Linear Forecasting

Linear forecasting, also known as trend analysis, involves fitting a linear regression model to historical data. It assumes a linear relationship between time and the variable being forecasted. The formula for a simple linear forecast is:

[ Y_t = a + bt ]

Here, (Y_t) represents the forecast for time (t), (a) is the intercept, (b) is the slope of the line, and (t) is the time period.

Here is some pseudocode for how it would appear in an Microsoft Excel/Google Sheets:

FORECAST(x, data_y, data_x)

Ensemble Methods and Seasonality

Additionally, always consider the option to combine methods, often known as ensemble techniques. Combining a traditional forecasting method with other well-known methods can result in a more performant method for your use case. It is also worthwhile, for any method, to consider your organization’s seasonality indexes. Seasonality indexes are modifiers that scale the data points to ratios that appear often in previous periods. Seasonality can affect Quarters, Months, Weeks, or even Days, so a seasonality analysis is always recommended.

2. Ease of Use

Traditional forecasting methods are easy to implement and require minimal computational resources. This ease of use can be a significant advantage in situations where time and resources are limited. With a few lines of code or even simple spreadsheet functions, you can quickly generate forecasts using these methods.

In contrast, AI-based forecasting models often demand extensive data preprocessing, hyperparameter tuning, and training times that may not be feasible for every project. Additionally, these advanced models often require substantial computational power, making them less accessible for smaller businesses or organizations with limited resources.

3. Quick Setup

Another crucial advantage of traditional forecasting methods is the speed at which they can be set up. When a new forecasting task arises, data scientists can swiftly implement less complex techniques and generate initial forecasts. This rapid turnaround can be invaluable in fast-paced business environments where decisions need to be made quickly.

AI methods, on the other hand, often require lengthy development cycles. From collecting and cleaning data to building, training, and fine-tuning complex models, the process can be time-consuming. This delay could lead to missed opportunities or ineffective decision-making.

The Role of Advanced AI Methods

While traditional forecasting methods offer several advantages, it’s important to acknowledge the specific scenarios where advanced AI methods shine. These situations typically involve complex and non-linear relationships within the data, large datasets, or the need for highly accurate predictions. Machine learning techniques like neural networks, ARIMA, or ensemble methods may be better suited for these challenges.

Conclusion

In the realm of forecasting, traditional methods such as Moving Average, Exponential Moving Average, and Linear Regression Forecasting continue to play a vital role. Their simplicity, ease of explanation, ease of use, and quick setup make them the preferred choice in many practical scenarios. These methods provide reliable forecasts that can guide decision-making and resource allocation effectively.

That said, the decision to use traditional or AI methods should be based on the specific characteristics of the data and the goals of the forecasting task. While AI methods offer advanced capabilities and can handle complex data relationships, they come with greater complexity, resource requirements, and potential pitfalls. Striking the right balance between traditional and AI forecasting methods is essential to ensure accurate and actionable forecasts while managing constraints.

As data professionals, our role is not only to employ the latest technologies but also to choose the most appropriate tools for the task at hand. In many cases, simplicity and transparency remain crucial. Traditional methods of forecasting have been well-known and trusted for many years. Always consider the importance of ease of explanation, ease of implementation, and ease of use when considering a new forecasting project.

What’s going on right now

What I’m building right now:

🚧 A small web app to report and track Virginia Beach tides, written in Go.

What I’m drinking right now:

🍺 Three Notch’d 40 Mile

What I’m reading right now:

📚 Ghost in the Wires by Kevin Mitnick

What I’m watching right now:

📺 Only Murders in the Building 📺 Twisted Metal

What I’m playing right now:

🎮 Starfield

What I’m learning right now:

🎓 🎸 Learning Never Let You Go by Third Eye Blind.
🎓 neovim
🎓 GoLang
🎓 Calculus refresher material! Let’s learn Differential Calculus (Again)
🎓 ChatGPT LLM Prompting for best effectiveness.