Technical details
The API, the guarantee behind every answer, the full benchmark results, and the supported scope.
API
One request, one answer with its uncertainty.
{
"version": 3,
"dataset": "nyc-taxis-2024",
"budget": {
"max_percent": 0
},
"filter": {
"column": {
"path": [
"PULocationID"
]
},
"predicate": {
"op": "in",
"values": [
{
"type": "i64",
"value": 132
},
{
"type": "i64",
"value": 138
}
]
}
},
"group_by": {
"path": [
"PULocationID"
]
},
"aggregates": [
{
"op": "avg",
"column": {
"path": [
"total_amount"
]
}
}
]
}{
"stage": "initial",
"estimate_id": "est_fdc7da8a13c20fd83cd0fa75a1484219",
"rows": [
{
"group": [
132
],
"aggregates": [
{
"value": 74.2148,
"exact": false,
"bounds": {
"kind": "compatible_range",
"lower": 57.4596,
"upper": 95.4561
}
}
]
},
{
"group": [
138
],
"aggregates": [
{
"value": 69.0424,
"exact": false,
"bounds": {
"kind": "compatible_range",
"lower": 50.3505,
"upper": 90.1809
}
}
]
}
],
"refinement": {
"available": true,
"reason": "more_information_available",
"cumulative_consumed_percent": 0
}
}Zones 132 and 138 are JFK and LaGuardia. At 0% their bounds overlap, so the question is still open, and the response says more reading can help: refinement.available is true. When more reading cannot improve an answer, as for a distinct count over a whole table, it is false and reason says why.
Refining an answer
Send the answer's estimate_id with a higher budget. Reading resumes where it stopped. At 10% the bounds separate: JFK costs more. Each range contains the true average computed on the source, $76.53 and $66.45.
{
"version": 3,
"estimate_id": "est_fdc7da8a13c20fd83cd0fa75a1484219",
"dataset": "nyc-taxis-2024",
"budget": {
"max_percent": 10
}
}{
"stage": "refined",
"estimate_id": "est_fdc7da8a13c20fd83cd0fa75a1484219",
"rows": [
{
"group": [
132
],
"aggregates": [
{
"value": 76.1222,
"exact": false,
"bounds": {
"kind": "compatible_range",
"lower": 74.9815,
"upper": 78.6938
}
}
]
},
{
"group": [
138
],
"aggregates": [
{
"value": 66.4124,
"exact": false,
"bounds": {
"kind": "compatible_range",
"lower": 66.173,
"upper": 67.2798
}
}
]
}
],
"refinement": {
"available": true,
"reason": "more_information_available",
"cumulative_consumed_percent": 9.976800910960774
}
}MCP
The same operations, exposed as read-only MCP tools:
- list_datasets
- describe_dataset
- query_dataset
- improve_answer
Answers
Three kinds of answer, each labelled.
- Exact
- The answer is the true value, and the response says so.
- Deterministic bounds
- Quantiles, top-k counts, SUM, COUNT and AVG on a single table. The true value lies between the bounds, guaranteed. Spending budget tightens them.
- 95% confidence interval
- COUNT(DISTINCT), aggregates over joins, and any aggregate the response labels with this kind of bound. The interval holds the true value with 95% confidence. On a whole-table distinct count, the response states that more reading cannot improve it.
Statistics prepared at ingestion
During ingestion, Hilbium prepares its own statistics about the data. A 0% budget means no source row is read while that query runs. They serve every supported question: no materialized views, rollups or marts to design for each new one.
Benchmarks
Full benchmark results.
ClickBench
26 of ClickBench's 43 queries map to the API. Every answer below is checked against the same query run on the source data, at a 0% read budget and, for top 10 queries, at 25%.
- Q0Exact
COUNT(*)
< 1 ms
99,997,497
- Q1Exact
COUNT(*), filtered
1 ms
630,500
- Q2Exact
SUM, COUNT(*), AVG
2 ms
- Q3Exact
AVG
< 1 ms
- Q495% confidence interval
COUNT(DISTINCT)
1 ms
−0.64% from the true value, observed
- Q595% confidence interval
COUNT(DISTINCT)
1 ms
+1.25% from the true value, observed
- Q6Exact
MIN, MAX
1 ms
- Q7Exact
COUNT(*), filtered and grouped
1 ms
18 groups
- Q8Not supported
COUNT(DISTINCT), grouped by region
Grouping by a numeric column without discrete values
- Q9Not supported
Four aggregates, grouped by region
Grouping by a numeric column without discrete values
- Q1095% confidence interval
COUNT(DISTINCT), filtered, grouped by phone model
17 ms
Intervals contain the true counts; right top 10 groups from a 25% budget
- Q11Not supported
COUNT(DISTINCT), grouped by two columns
7,304 potential groups, over the 2,048 limit
- Q12Top 10, bounded counts
Top 10 search phrases, filtered
5 ms
10 of 10 right at 0%; ranking settled at 25%
- Q13Not supported
COUNT(DISTINCT), grouped by search phrase
Grouping by high-cardinality strings
- Q14Not supported
COUNT(*), grouped by engine and phrase
Grouping by high-cardinality strings
- Q15Top 10, bounded counts
Top 10 users
2 ms
8 of 10 right at 0%, 9 of 10 at 25%
- Q16Not supported
COUNT(*), grouped by user and phrase
Grouping by high-cardinality strings
- Q17Not supported
COUNT(*), grouped by user and phrase, unordered
Grouping by high-cardinality strings
- Q30Not supported
Three aggregates, filtered, grouped by engine and IP
Grouping by a numeric column without discrete values
- Q31Not supported
Three aggregates, filtered, grouped by event and IP
Grouping by a numeric column without discrete values
- Q32Not supported
Three aggregates, grouped by event and IP
Grouping by a numeric column without discrete values
- Q33Top 10, bounded counts
Top 10 URLs
4 ms
10 of 10 right at 0%; 5th and 6th still swapped at 25%
- Q34Top 10, bounded counts
Top 10 URLs, with a constant column
4 ms
10 of 10 right at 0%; 5th and 6th still swapped at 25%
- Q35Top 10, bounded counts
Top 10 client IPs
3 ms
5 of 10 right at 0%, 6 of 10 at 25%
- Q36Top 10, bounded counts
Top 10 URLs, filtered on one site and month
29 ms
Undecided at 0%; 8 of 10 right at 25%
- Q37Top 10, bounded counts
Top 10 titles, filtered on one site and month
28 ms
Undecided at 0%; 8 of 10 right at 25%
hits table: 99,997,497 rows, 105 columns, 13.73 GiB of Parquet. Top 10 queries, written in SQL as ORDER BY COUNT(*) DESC LIMIT 10, are asked as an approximate top-k; every count returned and checked fell within its bounds. Observed errors come from this run; the guarantee is the interval. Times are median server times at a 0% budget over fifteen runs, measured on 2026-11-06.The other 17 rely on operations outside the API's scope:
- Q19, Q23–Q26: return rows, not aggregates
- Q20–Q22: LIKE
- Q27, Q28: string length and regular expressions
- Q18, Q29, Q39: expressions in grouping or aggregates
- Q38, Q40–Q42: OFFSET
A quantile closing to exact
| Budget | Bounds | Read | Time |
|---|---|---|---|
| 0% | 1,085 – 1,996 | 0% | 4 ms |
| 1% | 1,085 – 1,996 | 0.898% | 12 ms |
| 5% | 1,085 – 1,996 | 4.915% | 12 ms |
| 25% | 1,368 – 1,639 | 24.978% | 22 ms |
| 50% | 1,637 – 1,639 | 49.963% | 36 ms |
| 100% | exact: 1,638 | 90.641% | 54 ms |
approx_quantile(ResolutionWidth, 0.5) WHERE AdvEngineID <> 0, over 630,500 matching rows. The bounds close around the true median, 1,638, which every step contains. At a 100% budget the answer is exact after reading 90.6% of the rows: the engine stops once nothing left to read can change it. Measured on 2026-11-06.Raising a budget without reading twice
On the NYC Taxi query of the home page, raising one answer's budget from 0% to 1%, 5% and 25% gave the same bounds and the same share read as asking each budget directly. Nothing already read is read again.
Scope
What works today, and what comes next.
Available today
- Data
- Structured tables, ingested from Parquet files in your data lake, nested fields included. JSON Lines files are converted to Parquet first.
- Column types
- Booleans, 64-bit integers, 64-bit floats, decimals up to 18 significant digits, timestamps and strings.
- Aggregates
- COUNT, SUM, AVG, MIN, MAX, quantiles, the k most frequent values (k up to 100), and approximate COUNT(DISTINCT).
- Filters
- Equality and comparisons, IN, BETWEEN, null and boolean tests, combined with AND, OR and NOT across any number of columns, at any depth.
- GROUP BY
- Up to 2,048 groups. Continuous numeric columns are grouped into ranges or calendar periods, not by individual value.
- Joins
- Equi-joins across datasets: inner, left, right, full, semi and anti, or several inner joins combined. COUNT, SUM and AVG, grouped by columns of any joined table, each answered with a 95% confidence interval. A single inner join without GROUP BY tightens as the budget grows, until exact; other joins are answered at a 0% budget.
- New data
- Appended incrementally, without a full rebuild. Updates and deletes come later.
- Interface
- A JSON API, and the same operations as MCP tools.
Roadmap
- More types and aggregates
- Broader column types and aggregate functions.
- Updates and deletes
- Changing or removing existing rows without a full rebuild.
- Team scan policies
- Read limits and spending budgets set per team, so AI assistants and agents explore your data with controlled autonomy.
- Controlled sharing
- Share a dataset with partners, customers or other teams, and set how much of the source data their questions can read. They get aggregated answers with bounds, never rows.
- Your own cloud account
- The managed service, deployed inside your infrastructure.
Joins match rows on equal keys only.
Fit
Is it for you?
A good fit
- Ad hoc exploration of large tables.
- Analytical questions asked through AI assistants and agents, many and hard to predict.
- Filters that keep changing, so predefined views are hard to maintain.
- Questions where a bounded answer now is worth more than an exact one later.
Outside today’s scope
- Joins on conditions other than equal keys.
- GROUP BY over more than 2,048 groups.
- Tables whose rows are updated or deleted in place.
- Data that must stay in your infrastructure: deployment in your own cloud account is on the roadmap.