AI superforecasters outperform Metaculus superforecasters before 01.01.2028

Forecaster A: Historical Base Rates

Historically, AI has consistently made significant gains in specific areas like games and certain analytical domains. Given the rapid progression in AI model capabilities, the historical base rate for AI catching up with and surpassing human expertise in forecasting by a target date is moderate to high. Therefore, I estimate a probability of 65%.

Forecaster B: Current News and Evidence

Evidence suggests that AI models are closing the gap with human superforecasters, with recent tracks indicating they might surpass human performance by mid-2027. Given the latest trends and enhancements in AI forecasting models, the probability of this event occurring is reasonable. Thus, I estimate a probability of 70%.

Forecaster C: Incentives and Game Theory

Various players in the field have strong incentives to develop forecasting AI that surpasses human capabilities, not only for prestige but also for potential financial gains. Considering the competitive landscape, these incentives are likely to accelerate progress. As such, I place the likelihood at 60%.

Forecaster D: Quantitative/Statistical Reasoning

The data shows rapid improvement in AI forecasting capabilities, suggesting a projected crossover by mid-2027. Given these extrapolations and statistical models, the probability seems favorably high. I would assign a 65% probability.

Forecaster E: Devil’s Advocate

While current trends are promising, there are potential roadblocks, such as unforeseen technical challenges or overhyped projections. Additionally, human forecasters may also improve or adapt. Thus, introducing skepticism, I suggest a lower probability of 55%.

Aggregated Probability

Considering the independent probabilities and reasoning, the aggregated forecast results in a final estimated probability of 63% that AI will outperform Metaculus superforecasters by January 1, 2028.

Based on the provided information, there are several key factors supporting a high probability of AI superforecasters outperforming Metaculus superforecasters before January 1, 2028:

1. Trends and Projections: The trend data from FutureEval suggests that AI bots are rapidly closing the gap between community and human pro forecasters, with a projected crossover by mid-2027.

2. Performance Improvements: Recent tournaments and independent benchmarks indicate AI models are achieving parity with human superforecasters in specific contexts, and some trends hint at outperforming them.

3. Advances in AI Models: Continuous releases and improvements in AI models (Gemini 3.1 Pro, GPT-5) show notable progress and suggest continuing advancements will enhance forecasting capabilities.

4. Ensemble Methods: Ensembling strategies are proving effective in achieving results on par with human crowds, suggesting further potential when optimized.

5. Academic and Independent Research: Studies demonstrate substantial improvements, with AI models closing the performance gap and some studies already showing no statistical significance between AI and top human forecasters on certain benchmarks.

Overall, given the rapid advancements in AI and projected performance improvements, a 65% probability seems justified, acknowledging the remaining uncertainty in forecasting innovation pace, unforeseen challenges, or changes in evaluation metrics.

Clarify the Event:

The event forecasts that AI models designed for superforecasting will outperform human superforecasters on Metaculus before January 1, 2028. This means consistently achieving better accuracy across various domains.

Estimate the Historical Base Rate:

Historically, AI in certain domains (chess, Go) has achieved superhuman performance. AI model progress has shown rapid advancement in predictive tasks. However, in domains requiring nuanced judgment, human expertise often remains significant.

Break the Problem into Independent Factors:

1. Current AI Capabilities: As of mid-2026, AI is closing the gap but hasn't surpassed human superforecasters.
2. Trends in AI Improvement: Projections suggest AI could exceed human performance by mid-2027.
3. Model Enhancement: Rapid improvements in AI models (Gemini 3.1, GPT-5.5) show potential for surpassing humans.
4. Integration and Hybrid Strategies: Human-AI ensembles already show enhanced accuracy.
5. Metaculus and AI Integration: Successful AI integration in forecasting is progressing, but systemic acceptance of surpassing humans is crucial.

Estimate Each Factor Separately:

1. Current AI Capabilities: 60% (showing strong progress, near parity in some cases).
2. Trends in AI Improvement: 75% (consistent trend suggests probable human surpassing by mid-2027).
3. Model Enhancements: 80% (continuous updates and advancements are highly likely to boost performance).
4. Integration Strategies: 70% (likely improved results through hybrid strategies and model enhancements).
5. Metaculus Acceptance: 65% (future evaluations suggest ongoing tracking and potential acceptance).

Recombine Into an Overall Probability:

Aggregating the weighted probabilities, considering AI systems' exponential learning curves and advancements, the overall probability is around 75%.

Check for Overconfidence:

Acknowledging uncertainty factors such as unforeseen advancements in AI, changes in Metaculus testing methodologies, and unexpected human forecasting improvements could impact the probability.

Imagine Why You Could Be Wrong:

1. Human Factor: Human forecasters could innovate, applying new techniques boosting accuracy.
2. Systemic Changes: Metaculus might change evaluation methods, affecting comparative results.
3. Technological Plateau: AI advancements might slow unexpectedly, delaying surpassing humans.
4. Regulatory Changes: New rules around AI usage might hinder model deployment or development.

Prior Probability

Initially, the base expectation for AI exceeding top human forecasters would have been lower due to the challenges involved in matching human intuition and expertise in forecasting domains. However, ongoing advancements in AI and trends from other domains support a reasonable expectation that AI could eventually outperform humans.

New Evidence

The provided document shows:

1. Trend Analysis: As of mid-2026, AI bots on Metaculus are rapidly improving, although not yet surpassing top human forecasters. There are projections and future expectations that suggest a crossover by mid-2027.

2. Performance Improvements: AI models like Gemini 3 Pro and GPT-5 have shown significant improvement, narrowing the performance gap with humans (e.g., decreasing head-to-head score differentials).

3. Independent Benchmarks: Other independent assessments (ForecastBench) claim AI models are almost statistically indistinguishable from human superforecasters in some contexts.

4. Ensemble and Hybrid Models: When AI predictions are combined through ensemble methods or augmented with human forecasts, the accuracy closely matches or exceeds human-only performance.

Likelihood Ratios

Given AI's rapid improvement trends, the likelihood that it will outperform humans continues to rise. Historical data and projections from credible sources like Metaculus suggest that AI is on track to meet or exceed these expectations by the target date.

Posterior Probability

Considering the strong trajectory of advancements and frequent performance improvements, the probability that AI superforecasters will outperform Metaculus superforecasters by January 2028 is substantial. Given current trends and data, a 70% probability reflects both the tangible progress of AI and the inherent uncertainties of future developments. This forecast accounts for the potential for unforeseen technological breakthroughs or impediments.

Overall, with continuous AI model enhancements and supportive empirical evidence, there is confidence in a favorable outcome by the specified resolution date.

The forecast is centered around the likelihood that AI superforecasters will outperform Metaculus superforecasters by January 1, 2028. The evidence highlights several key points:

1. Current Performance and Trends: AI models are rapidly improving and have been closing the performance gap with top human forecasters, as seen in tournaments and independent benchmarks.

2. Projected Progress: Meta forecasts from platforms like FutureEval suggest that AI might surpass human superforecasters by mid-2027.

3. Technological Advances: Continual advancements in AI technology (e.g., newer models like GPT-6, Claude Opus) are likely to enhance forecasting abilities.

4. Emerging Parity: Ensemble models and hybrid strategies have already achieved parity in some contexts, suggesting that full outperforming is feasible within the given timeframe.

5. Potential Challenges: Factors like changes in tournament structures, AI model deployment, and the inherent complexity of forecasting across diverse domains could impact AI performance.

Considering these aspects, the evidence supports a reasonable likelihood of 65%. There's optimism based on trends but also acknowledgment of potential risks and uncertainties that could delay projected achievements.