ODTQA-FoRe: An Open-Domain Tabular Question Answering Dataset for Future Data Forecasting and Reasoning

Stop Asking LLMs to Predict the Future

Most AI models are great at looking in the rearview mirror. They can scan a table and tell you what the average house price was last June. However, if you ask them what the price will be next June, they often hallucinate. They struggle with basic arithmetic. Most current systems are built for static knowledge retrieval. They answer questions based on data that already exists. They are not built for time-series forecasting (modeling trends over time).

This gap is where the ODTQA-FoRe research sits. The authors argue that LLMs excel at understanding natural language. Yet, they lack dedicated mechanisms to model dynamic, numerical time-series data. To bridge this, they introduce a new task: Open-Domain Tabular Question Answering for Future Data Forecasting and Reasoning (ODTQA-FoRe). They are not just asking the model to guess a number. They ask it to retrieve historical data, predict future values, and then reason about those predictions.

The Problem

Current open-domain tabular QA (ODTQA) methods typically rely on Retrieval-Augmented Generation (RAG) to pull facts from a database. This works well for historical queries. It breaks down when the query requires a forecast. As shown in, a user moves from a simple historical query (Q1) to a forecasting query (Q2).

They might then ask a complex reasoning query (Q3) to compare predicted trends.

Existing approaches fail for two reasons. First, LLMs are trained on static text. This makes them fundamentally poor at the mathematical heavy lifting required for forecasting. Second, in an open-domain setting, the system does not know which table holds the history. It must autonomously navigate a massive database to find the right sequence. Relying on an LLM to do both retrieval and math leads to high error rates.

How It Works

The authors propose TimeFore, an LLM agent-based framework. It treats the problem as a collaborative workflow. Instead of one model doing everything, they decompose the task into three specialized roles :

1. The Retriever: This agent handles data acquisition. It uses the LLM to summarize the user's query into a canonical table caption. It then uses the BM25 algorithm (a probabilistic text retrieval method) to find the most relevant table. Once identified, the Retriever generates and executes SQL queries. It uses an execution-feedback loop to refine queries up to 25 times if they fail.
2. The Forecaster: This is the critical architectural pivot. Rather than forcing the LLM to calculate trends, the Forecaster uses function-calling to invoke specialized models. It uses a pipeline involving TimesNet for data imputation (filling in missing values) and TimeXer for generating a 12-month price forecast. This offloads the numerical work to models designed for math.
3. The Analyzer: The final agent takes the retrieved history and the forecast. It synthesizes them into a final answer. It uses a BERT-based classifier (a model used for text categorization) to determine the query type. It then applies tailored prompts to ensure the response is standardized.

Numbers

The core takeaway is that specialized tools beat general intelligence for math. The paper reports that TimeFore significantly outperforms "Vanilla" LLM baselines. For example, using Qwen3 30B, the framework improved Forecast-based Reasoning F1 scores (a measure of predictive accuracy) from 24.87% to 60.25% [Table 3]. This means the system became more than twice as effective at reasoning about future trends.

The authors also conducted an error decomposition study. The results in are telling.

The primary bottleneck is not the SQL generation or the table retrieval. The bottleneck is the accuracy of the forecasting itself. Providing the Analyzer with "Golden Future" data (perfect ground-truth forecasts) yielded the largest performance gains. This suggests that if you fix the Forecaster, the reasoning pipeline becomes very robust. Regarding the forecasting task, specialized models like TimeXer achieve a much lower Mean Relative Error (MRE) than general-purpose LLMs like GLM 4.5 Air [Table 2].

What's Missing

The framework is impressive, but there are several gaps. First, the scope is narrow. The ODTQA-FoRe dataset is restricted to the real estate vertical. The success of the Forecaster depends on the TimeXer model. This model may not generalize to highly volatile domains without significant retraining.

Second, the forecasting engine is "blind" to exogenous variables (external factors like interest rates or policy). The current pipeline only looks at the historical price sequence. The authors admit the model does not yet incorporate these external drivers. This is a major limitation for serious financial applications.

Finally, the dataset relies on template-based generation. Although they used an LLM to rewrite queries for diversity ([Appendix A.3]), the logic is still anchored to structured templates. This might hide edge cases where user intent is highly ambiguous.

Should You Prototype This

Yes, if your problem involves time-series forecasting. If you are building a "predictive RAG" system for financial or inventory data, this multi-agent architecture is a solid blueprint. Do not try to make the LLM do the math. Use the Retriever $\rightarrow$ Forecaster $\rightarrow$ Analyzer pattern to delegate the heavy lifting to specialized tools.

Code is reportedly available at https://github.com/jensenw1/ODTQA-FoRe. If you prototype this, focus on the Forecaster. As the ablation studies in

show, your system's utility is capped by the accuracy of your time-series model.

Figures from the paper