UQO Working Paper No. 13· v1.2· November 2026· 74 pages
VQuant: An Agentic Large Language Model Architecture for Automated Quantitative Financial Analysis and Document Generation
A Specification Audited Against the Code, with Usage Evidence and an Observational Grounding Measure from the Deployed Instance
A production LLM agent for quantitative finance specified precisely enough to be checked: 245 tools, 50 code facts extracted rather than typed, and an unsupported-number rate of 0.89% [0.79, 1.01] in its persisted reports.
The language model plans and writes while deterministic code retrieves, computes and accounts. The specification is audited against one commit of the source code and against read-only records of the deployed instance, which exposed several places where the implementation departs from its own description. The grounding rate is observational and single-arm; the paired evaluation that would show what the tools add has not been run.
Simon-Pierre Boucher — contact@spboucher.ai
0.89% [0.79, 1.01]
Unsupported-number rate over 26,568 numeric tokens in 395 persisted reports scored against their own stored tool outputs (237 unsupported). Observational, single-arm and scored by a program without human annotation; it does not say what the tools add.
245 tools
Schema-defined tools in the catalogue at the audited commit, rather than the 237 of a source comment or the 213 of the repository's README; 3 catalogue tools have no handler and 2 tools named in the system prompt are absent from the catalogue.
50 / 33 / 0
Facts extracted by script from one fixed commit with file and line anchors, facts derived from them, and facts typed by hand.
33.6%
Share of provider-reported output tokens that the platform's cost ledger recorded in 265 logged calls, because it reads a field the provider does not populate; at the default model's list rate the window's cost is 3.54 times the ledger's.
0 of 3,414
Model calls on which the third context-management stage (adaptive summarisation) fired; its estimator ignores the 26,104-token tool catalogue that dominates every request.
3 calls, 53 s
Median recursion depth and median end-to-end latency over 1,062 completed requests (latency interquartile range 25–97 s); the Python sandbox succeeded in 205 of 277 executions (74%).
187 of 213
Continuation calls with non-zero thinking tokens in the provider's usage events, which refutes a conjecture raised during quality control that adaptive reasoning is silently disabled on continuation calls.
Abstract
Large language models write fluent financial prose, but a fluent number is neither retrieved nor computed. This paper specifies VQuant, a deployed web platform in which a reasoning-capable model orchestrates 245 schema-defined tools—market-data wrappers, web research, and a Python sandbox—through a recursive, batched, streaming loop, and audits that specification against one commit of the source code: 50 facts extracted with file and line anchors, none typed. Read-only exports from the instance (301 persisted sessions, 1,062 completed requests) replace earlier assertions with measurements: median recursion depth 3, median latency 53 seconds, and a cost ledger that captures 33.6% of the output tokens the provider bills. Scoring every persisted report against its stored tool outputs gives an unsupported-number rate of 0.89% [0.79, 1.01], an observational, single-arm quantity. The pre-registered paired evaluation has not been run; the paper reports its analysis plan and the contrast it could detect.
JEL: C45, C63, C88, O33
Key Results
A constants-level specification of a production loop
The streaming transport, the recursive tool-calling procedure, the batching policy, prompt-level routing over 245 tools, the three-stage context procedure and the single-call computation convention are stated as algorithms with their extracted constants, including where the implementation departs from its own intent.
A re-runnable audit
Every quoted fact is produced by a script that refuses to run unless the code tree and file hashes match a committed manifest; the generated tables and the verbatim prompts are part of the package.
Measured behaviour of the deployed instance
Recursion depth, tools per round, batching, sandbox success, latency, ledger and provider token counts, one fully traced persisted session and the observational unsupported-number rate replace earlier behavioural assertions.
Defects reported as findings
The audit exposed a re-send cap that overrides the execution-time limit, a summarisation stage that has never fired, a cost ledger that under-records output tokens, and a sandbox that inherits the server's credentials under anonymous use; the paper lists the engineering changes that follow.
A pre-registered evaluation protocol with a statistical analysis plan
Primary endpoint, paired design, ground truths, inference and minimum detectable differences are defined; the protocol has not been executed and the reasons (cost at list price, no staging deployment, no task-level ground truth) are stated.
Quality control
This paper went through a multi-stage quality control: 6 quality-control reports and 79 points checked. The version presented on this page is the revised manuscript. Scores are on a 0–100 scale and are a diagnostic of the manuscript, not an acceptance probability.
Score after revision
54.9/100
38.1 before quality control · +16.8
Quality control dated November 4, 2026
- Quality-control decision
- claims not supported as written → reframed and revised
- Scientific integrity
- PASS
- Reproducibility
- R1reproduced from documented analysis data
- Maturity
- Cneeds new analysis or data
- Quality-control reports
- 6
- Points checked
- 79
Scores by dimension
Before quality controlAfter revision
- Originality & contribution37.5 → 42.0
- Scientific rigor49.2 → 68.0
- Identification & methodology30.0 → 45.0
- Data & reproducibility32.5 → 70.0
- Robustness & validation10.0 → 22.0
- Writing70.8 → 74.0
- Figures & tables58.3 → 78.0
- Literature positioning48.3 → 72.0
Revised manuscript74 pages9 figures13 tables3 equations65 references29,516 words
What quality control changed
Quality control judged that the earlier version asserted behaviour it had not measured. The revised paper replaces those assertions with facts extracted from the code and with read-only records of the deployed instance, reports the defects this exposed, and adds an observational grounding rate. It still contains no paired evaluation and makes no comparative claim.
- System constants: described from documentation → 50 facts extracted by script from one commit with file and line anchors (for example 27 sandbox packages under 89 names).
- Cost accounting: presented as reliable → reported as a defect; the ledger records 33.6% of the output tokens the provider bills.
- Context management: three stages described as operating → the third stage never fired in 3,414 model calls.
- Behaviour: asserted → measured on 301 sessions and 1,062 completed requests (median depth 3, median latency 53 seconds, sandbox success 74%).
- Worked example: a hypothetical walk-through → one persisted session traced from the record.
- Grounding: no measurement → an observational, single-arm unsupported-number rate of 0.89% [0.79, 1.01]; the paired tools-on/off evaluation is stated as not run.
Limitations stated in the paper
- No paired evaluation: what the tools add relative to a model without them is not measured, and nothing in the paper supports a comparative claim.
- The observational rate is automated and single-arm, with no human annotation; several scoring conventions bias it downward and one biases it upward.
- Executed code and tool inputs are not persisted, per-call model identity is not logged, and requests before 2026-10-02 may have run earlier code.
- The usage figures describe one deployment with almost entirely anonymous, unprompted traffic, and the platform's repository is private.
Data
Source code at one audited commit
The platform's TypeScript and Python code at commit e2a148a (2026-10-02), the deployed state on 2026-11-04: a React front end, an Express back end and an isolated Python layer with 27 third-party packages bound under 89 names. The market-data provider is wrapped by 249 typed functions over 233 endpoint paths.
Read-only exports from the deployed instance
Application logs (183 daily files, 2025-10-21 to 2026-11-03), a copy of the session store (301 sessions, 300 of them anonymous; 445 turns; 443 shared reports, 397 with stored tool results) and 265 provider-side usage events. No paid call was made.
Persisted reports scored for grounding
395 scorable reports with their full stored tool outputs: 82.8% of numeric tokens match a stored number verbatim and 16.4% are derivable from stored numbers; 314 reports contain no unsupported number.
Methodology
Design-science framing
The paper presents an artifact, states the problem it addresses and specifies its construction; it is explicitly not an evaluation and makes no comparative claim about accuracy, latency or cost.
Script-extracted code audit
Constants and counts are extracted by script from a fixed commit, each with the supporting file and line, and an appendix lists every statement of the paper against its source.
Usage descriptives
Recursion depth, tool counts, failures, latency and token accounting are computed from logs, the session store and provider usage events, for the full window and for the window in which the audited commit is certain to have been deployed.
Observational grounding measure
Each numeric token of a persisted report is labelled verbatim, derived or unsupported against the stored tool outputs, with Wilson and report-clustered bootstrap intervals.
Proposed paired protocol (not executed)
Retrieval, computation and report task families with independent ground truth, ablations led by a tools-disabled arm, baselines, controls and a statistical analysis plan; at 50 tasks per arm the minimum detectable difference is 23 percentage points.
Reproducibility
- The audit is re-runnable: the extraction script checks the code tree and the hashes of the 42 files it reads against a committed manifest before producing the tables and facts quoted in the text.
- The manifest, the generated tables and the verbatim prompts are part of the paper's package; the platform's own repository is private, so the audit is re-runnable only by someone with access to it.
- The design is reproducible from the paper (algorithms, constants, prompts, inter-process contract); the outputs are not, because sampling is not seedable, the Python code is authored by the model, and the Monte Carlo routine has no fixed seed.
- Executed Python source is not persisted, so figures in any report cannot be regenerated and numbers the sandbox computed from self-fetched data cannot be checked from the record.
Cite this paper
Boucher, S.-P. (2026). VQuant: An Agentic Large Language Model Architecture for Automated Quantitative Financial Analysis and Document Generation. UQO Working Paper No. 13, Université du Québec en Outaouais.
@techreport{boucher2026wp13,
author = {Simon-Pierre Boucher},
title = {VQuant: An Agentic Large Language Model Architecture for Automated Quantitative Financial Analysis and Document Generation},
institution = {Université du Québec en Outaouais, Département des sciences administratives},
number = {13},
type = {UQO Working Paper},
year = {2026},
month = {nov},
url = {https://www.spboucher.ai/research/wp13},
}More in Finance & Economics