Bulk downloads paused for some clients. Requests identifying as "AlphaHound research importer" are blocked while we rebuild the archive, because they took over half of all download capacity. Bulk downloads are also paused while we roll out v3.1 (a new format with smaller files) for "polytaker-source-audit", a python-requests client on one IPv6 network and a client sending a browser user agent from one server. Please contact us for a mirror arrangement: Discord or X.
Polymarket Orderbook Archive archive.pendulumflow.com

Battle test: compare v3 and v3.1 yourself

v3 is being replaced by v3.1. We have rebuilt the order-book archive as v3.1. It is split by market category, so you can download only what you need, and it is served from a faster cache.

v3.1 already covers 27 October 2026 onward. The rebuilt history from 31 August to 26 October 2026 is being published over the next few days.

When all of that history is published (we expect around 5 November 2026), v3 downloads will be switched off. Six v3 hours will stay online on this page so you can check v3.1 against v3.

v3 also has hours that v3.1 does not yet have: 18 Aug 06:00 to 31 Aug 12:00 UTC (2026). We are rebuilding those in v3.1 next, together with earlier hours from 15 August that a contributor shared with us. Until they are published, those v3 hours stay online.

Using v3 today? See what changed in v3.1 and our step-by-step guide to switching for the new file layout and how to change your scripts or DuckDB queries. Problems or questions: feedback.

We have rebuilt the order-book archive as v3.1. So that you can check it against v3, we are keeping six v3 hours online after v3 is switched off. Each one has a v3.1 twin. The hours below were picked to cover the cases where a rebuild is most likely to go wrong.

hour (UTC)why this hour
27 Oct 2026, 14:00Where one of our build runs handed over to the next. This hour's opening order book was rebuilt from earlier data rather than carried over from 13:00. Compare its opening state with the hour before (test 8).
28 Oct 2026, 00:00Midnight UTC, and the first hour after we changed which of our recorders' copies is preferred when two copies of a message differ. We run 3 to 5 recorders.
28 Oct 2026, 17:00The hour where the most rows were removed at the start because the previous hour already had them (18,359 rows). Both v3 and v3.1 remove them.
28 Oct 2026, 19:00An hour checked with every one of our verification checks, not a sample.
29 Oct 2026, 00:00The busiest hour in the overlap: 180 million v3 rows, 2.0 GB.
29 Oct 2026, 18:00The quietest: a Polymarket disruption, no trades all hour, and about 1/60 of the usual updates. The v3 page already marks it as an incident.

What is different in v3.1, and what you should see

The v3.1 format page is the full reference. Where this summary and that page differ, that page is right.

  • Fewer rows; the values match. Polymarket sends many updates twice, once for each side of a market, as exact mirror images. v3 keeps both copies; v3.1 keeps one, and every dropped mirror row can be rebuilt from the provenance file and the hour's token map. v3.1 also drops order books that repeat the previous one exactly. Those repeat books are dropped on purpose and cannot be rebuilt.
    • Test 1 shows that, for every event type, the v3 rows equal the v3.1 rows plus the two dropped counts in the manifest (mirror_rows_dropped and book_rows_dropped, at its top level).
    • Test 10 shows that every row v3.1 stores has the same values as its v3 row.
  • If you read only the lean files, you see one side of each mirrored pair. Rebuild the other side from the provenance file and the token map (below).
  • Split by market category, and sometimes further. One v3 file per hour becomes, per category such as crypto, politics or sports, a lean file and a matching .provenance.parquet. A busy (category, sub_tag) gets its own pair, and a large unit is split into parts .p1 … .p<N>. So a file name reads <hour>.<category>[.<sub_tag>][.p<N>][.provenance].parquet. The hour's manifest.json lists every file: start from it, and never guess file names. To rebuild mirror rows you also need the token map that its canonical_token_map.object names, under v3.1/maps/, which is not in the file list.
  • Columns moved.
    • Now in the provenance files: sequence, source_witness (which of our recorders the row came from), witness_set and arrival_skew. v3.1 keeps arrival_skew in microseconds but floors it to a whole millisecond.
    • timestamp_received is not stored. It is timestamp + recv_lag_ms, to the millisecond, rounded down. recv_lag_ms can be negative when the venue's clock is ahead of ours.
    • spread is not stored. It is CASE WHEN spread_differs THEN spread_raw WHEN event_type = 'best_bid_ask' THEN best_ask - best_bid END. Use CASE, not COALESCE, or a best_bid_ask row whose message had no spread field comes out wrong.
    • New columns: category, sub_tag, row_ordinal, recv_lag_ms and mirror_dropped.
  • Row order. Each lean file holds one run of rows per event type, in a fixed order (best_bid_ask, book, last_trade_price, market_resolved, new_market, price_change, tick_size_change). Within each run, rows are sorted by category, sub_tag, market, token, timestamp and row_ordinal. A token's full order is (timestamp, sequence, row_ordinal), and sequence is in the provenance file.
  • New files.
    • opening_state.parquet: every token's order book at the start of the hour. It goes by the venue's time, so rows received up to about 60 minutes after hh:00 can enter it. It is not what a live observer knew at hh:00; for a start with no lookahead, rebuild from rows with timestamp + recv_lag_ms ≤ hh:00, in that order. Its columns are on the v3.1 format page.
    • twin_check.parquet: counts and fingerprints for checking the rows you rebuild. Its columns and fingerprint (twin-check-v1) are on the v3.1 format page and in quickstart.py.
    • verify_report.json: our own checks on the hour.
  • Smaller downloads, except in a very quiet hour. The lean files alone are about 26–29 % of the v3 file's size, and everything together is 53–57 %. In the quiet hour (29 Oct 18:00), v3.1 is bigger than v3, about 136 MB against 81 MB: the opening state and twin check cover every token, however little it traded.
  • Served from a cache. v3.1 files come from a cache near you, and v3 files do not (test 9).

Before you start

  • You need DuckDB 1.5.5 or later. The first query installs the httpfs extension.
  • The busy hours are 1.5–2 GB in v3 and 0.8–1.1 GB in v3.1. Querying straight from the site works, but it makes many small reads, and the download server limits how fast one address can read, and answers HTTP 429 when you go over. Keep the retry settings in the script. For the busy hours, download the files first (test 7 does this) and point the queries at the local copies.
  • If you get HTTP 429 or 503, wait 30–60 seconds and run it again.

The tests

  • tests.sql (sha256 ae5aaec5558c55e47225a48307f98f55c0782335cf79fbf0b15bb3cf75934577): tests 1–6, 8 and 10, in DuckDB.
  • tests.sh (sha256 24e7483ca159d776aafae714415c7b07f50d348ecd55d9743996dbb62797473c): tests 7 and 9, with curl and sha256sum.

Run tests.sql in DuckDB. To test another hour, change the two SET VARIABLE lines at the top. Run tests.sh <day> <hh> for tests 7 and 9; it prints PASS or FAIL for each check and exits 1 if any fails.

The results below were measured on 29 Oct 2026, 18:00, except test 5, which needs trades and was measured on 27 Oct 2026, 14:00. For the other hours, the counts are in the table at the bottom.

1. Rows per event type

Every row's reconciles is true. An event type present on only one side would show as false.

event_typev3 rowsv3.1 rowsmirror twinsrepeat booksreconciles
best_bid_ask256,538129,274127,2640true
book441,509220,648218,6272,234true
market_resolved1,2131,21300true
new_market2,3452,34500true
price_change1,799,808899,951899,8570true
tick_size_change81681600true

(last_trade_price has no rows in this hour, on both sides.)

2. Schema diff, names and types

Exactly these differences, and no column in both has a different type:

  • only in v3: spread, timestamp_received;
  • only in the v3.1 lean files: category, sub_tag, recv_lag_ms, mirror_dropped;
  • in both v3.1 files (the join key): row_ordinal;
  • in both, but in v3.1's provenance files: sequence, source_witness, witness_set, arrival_skew;
  • only in v3.1's provenance files: is_primary_copy, spread_differs, spread_raw, twin_seq_delta, twin_recv_delta_ns, twin_arrival_skew.

The types, as DuckDB reads them: prices DECIMAL(9,4), sizes DECIMAL(18,6), times TIMESTAMP WITH TIME ZONE (UTC), market and token ids BLOB, and recv_lag_ms BIGINT. Every column's type is on the v3.1 format page.

3. Duplicates in the first 5 minutes

The same key on both sides, (event_type, sequence, source_witness): 0 in v3 and 0 in v3.1. Every v3.1 lean row has its provenance row. Rows the previous hour already had are removed at the hour's start in both archives; the manifest counts them under ab_accounting.<event type>.boundary_dropped.

4. Best bid and ask per minute, for the 3 busiest tokens

For each token, every minute's last best bid and ask are equal (differ = 0). In this quiet hour the busiest tokens have updates in only 7–8 minutes; in a normal hour expect all 60. This compares the token v3.1 stores; for the mirror token, rebuild its rows first. Test 10 checks every stored row.

5. Trades, row for row (27 Oct 14:00)

63,008 trades on both sides, with the same volume (7,683,795.856548). No trade is on one side only (only_in_v3 = only_in_v31 = 0), and no market differs. On 29 Oct 18:00 the test says "not applicable": that hour has no trades at all.

6. Time order

All four counts are 0:

  • v3 rows received outside the hour;
  • v3.1 rows received outside the hour (timestamp + recv_lag_ms);
  • v3.1 rows with no recv_lag_ms;
  • v3.1 rows where timestamp goes backwards within one token's run in a file.

v3 files are grouped by event type but not sorted by time within one.

7. Checksums

tests.sh reports PASS on all three lists:

  • v3: 4 of 4;
  • v3.1: 38 of 38 from SHA256SUMS.txt;
  • the manifest's own list: 36 of 36. The 2 extra lines in SHA256SUMS.txt are the manifest itself and the verify report, which the manifest cannot list.

8. The opening order book is where the previous hour left off, for the tokens tested

For each of the 3 tokens, book_as_of is the time of its last update stamped at or before 18:00, in the token's (timestamp, sequence, row_ordinal) order. The opening best bid and ask match that update: a price_change's own best bid and ask, or a book's highest bid and lowest ask. All three are equal: the last update for each is a price_change at 17:58:54–17:58:55 in the previous hour.

9. Cache

tests.sh reports PASS on all four reads; the v3.1 file is the manifest's first lean file.

  • v3.1: HIT, 206, about 0.05–0.1 s per 64 KiB range read.
  • v3: DYNAMIC (not cached), 206, 0.2–1.4 s.

HIT = served from the cache (fast); 206 = a partial download, which DuckDB uses.

After v3 is switched off, these six v3 hours stay online, still not cached.

10. Same values, row for row

Each of the 1,254,247 rows v3.1 stores is matched to its v3 row on (event_type, sequence, source_witness). Every shared column is compared, and all counts are 0:

  • unmatched rows;
  • time, market or token;
  • best bid, best ask and the book levels;
  • trade price, size, side, fee and transaction hash;
  • the market fields (id, outcomes, question, slug, tick sizes, resolution);
  • witness_set, and arrival_skew (v3's value floored to the millisecond);
  • timestamp_received (timestamp + recv_lag_ms against v3's value rounded down to the millisecond);
  • spread (by the CASE rule above, against v3's spread).

Coming to v3.1

Now shipped, in quickstart.py (README):

  • a one-line way to read any hour
  • a ready-made rebuild of the second token's rows
  • a simpler spread

More is coming soon:

  • a per-hour market and category index
  • hourly crypto settlement data (price to beat, final price, average)
  • Chainlink and Binance reference prices
  • the signed orders behind each fill, with links to their blockchain transactions
  • a public order-latency series
  • Polymarket's liquidity-reward parameters, published daily
  • each recorder's connection and gap log for every hour
  • a correction policy, with a changelog for every change that affects the data
  • an opening state with no look-ahead (only what a live observer could have known at the top of the hour)
  • the history back to August

Tell us which of these matter to you at /feedback.

Found a difference?

Tell us at /feedback and tag it battle-test. Please include:

  • the hour;
  • the test number;
  • the query's output;
  • your DuckDB version (SELECT version();).

The six pairs

hour (UTC)v3 rowsv3.1 rowsmirror twins droppedrepeat books droppedrows already in the previous hour (removed in both)tradesbook rows raw → keptv3 MBv3.1 MB (lean + provenance + rest)downloads
2026-10-27 14:00141,864,95371,205,70770,659,236101,17063,008139,030 → 78,732 (43.4 % cut)1,492807 (388 + 351 + 68)v3 file · v3.1 manifest
2026-10-28 00:00146,456,88273,691,94372,761,9912,9481,88165,155753,955 → 390,671 (48.2 % cut)1,613842 (414 + 345 + 83)v3 file · v3.1 manifest
2026-10-28 17:00148,693,60574,527,17974,166,4022418,35971,125327,101 → 169,203 (48.3 % cut)1,532871 (432 + 355 + 84)v3 file · v3.1 manifest
2026-10-28 19:00157,911,05479,142,87078,768,0161683,87977,906200,606 → 105,253 (47.5 % cut)1,634933 (464 + 387 + 82)v3 file · v3.1 manifest
2026-10-29 00:00180,050,33990,379,65089,248,324422,36596376,73710,525,323 → 5,124,936 (51.3 % cut)2,0181,113 (573 + 442 + 98)v3 file · v3.1 manifest
2026-10-29 18:002,502,2291,254,2471,245,7482,2342120441,509 → 220,648 (50.0 % cut)81136.3 (44.1 + 6.6 + 85.6)v3 file · v3.1 manifest

Download links and checksum lines, one block per hour:

2026-10-27T14

# v3/SHA256SUMS.txt
b13b8b348c07b09639f663610b4591981220e4a5877d9c9d314519c6ab9ba38b  2026-10-27/14/2026-10-27T14.parquet
38982d7722a9354a1e9d7f1050b9da561a491150f511d396d4a5658afca90257  2026-10-27/14/2026-10-27T14.witness_stats.json
4d5ceb8c56d00120bbd1801147910a1f9c9ef45bb2d2439df01ebd13ffd693bf  2026-10-27/14/manifest.json
# v3.1/SHA256SUMS.txt (manifest, verify report, opening state; the manifest lists the other files)
c172225051447323c4debe256ae9f10e02e8576f93ae4ecd8b36342d1082ba10  2026-10-27/14/2026-10-27T14.opening_state.parquet
ec028840c72c1e6329357d4effc93eeb5ab46d38859bf5b886354eb4963ed885  2026-10-27/14/2026-10-27T14.verify_report.json
efaeb06954fbda94eb508f40c0097361c3ab78d158a73e5758d3beb3f3dd4f60  2026-10-27/14/2026-10-27T14.manifest.json

2026-10-28T00

# v3/SHA256SUMS.txt
9ffd143658c7b3e6fad29b2b77e65d1bd5b8e334331f3c08835a1a211578a691  2026-10-28/00/2026-10-28T00.parquet
94a24ed52bf464286116c95e0062586d193aed0cef20fe3602ca35096fde4d5a  2026-10-28/00/2026-10-28T00.witness_stats.json
91cb5c196e136afde7c3d395b86adf9268d91bb44f2ed21ab4db7ca71bc4f310  2026-10-28/00/manifest.json
# v3.1/SHA256SUMS.txt (manifest, verify report, opening state; the manifest lists the other files)
7987c62bda440877cdac15f1b65820ae16dab4298a086367780e47cdbdf83dd2  2026-10-28/00/2026-10-28T00.opening_state.parquet
3c49055135835e249ca5ed4202c8b11416e79bbc3d7a12d059b6e9607d0ee4e0  2026-10-28/00/2026-10-28T00.verify_report.json
c0ce233859079d4422f738af0ac2f6e555afc975c78a4eff1afb90282fa3f0c7  2026-10-28/00/2026-10-28T00.manifest.json

2026-10-28T17

# v3/SHA256SUMS.txt
aafe0847f63167b1f3f470178d6df5ee4ad7bc20aaeb791c63b66f1b57e646bc  2026-10-28/17/2026-10-28T17.parquet
973a3d0fe707e324c4a0cd0817b37e453fb0d999cfcb9d98efccb03a6c52619c  2026-10-28/17/2026-10-28T17.witness_stats.json
245899e8805e7781621cf5d4d462bd16905c1c75b6fbe3888ce97d67c529f080  2026-10-28/17/manifest.json
a8d7087922bebf3455ceea5574c2db0f924deb073625b35c56327f8028ad2858  2026-10-28/17/2026-10-28T17.witness_delivered.json
# v3.1/SHA256SUMS.txt (manifest, verify report, opening state; the manifest lists the other files)
9323c9b05e8826d7815f501cfb91f65a18208c43e8ef1f3854641c1f9a07ca18  2026-10-28/17/2026-10-28T17.opening_state.parquet
49c99eeb93701b677155fd3193213d6a5abf59981b56d1c92d3543a77f206ca4  2026-10-28/17/2026-10-28T17.verify_report.json
201704a278c389348e0d13eb75c520fa3ec513c285dcbb4bef53a7ec99466d9a  2026-10-28/17/2026-10-28T17.manifest.json

2026-10-28T19

# v3/SHA256SUMS.txt
3472d9ddf80d8a51273af383c4592a9d94c0fb36ff7dc0c4bf492aebf86e7f31  2026-10-28/19/2026-10-28T19.parquet
aad778fae5a194da5bd09010e717f07744360d8bd79be598ee454b7b10c3def7  2026-10-28/19/2026-10-28T19.witness_stats.json
b03d0f7e30a3764706744aced1e670cb30470e93bf9bf26b92cf2a4cc41cd7c5  2026-10-28/19/manifest.json
7badb103f06fe027a8f54fc9cc48c24727a73bd0f63d876e36f77c42795e146d  2026-10-28/19/2026-10-28T19.witness_delivered.json
# v3.1/SHA256SUMS.txt (manifest, verify report, opening state; the manifest lists the other files)
b9c1a21eecad1da9abed79db7d2c08b40ea4340d7d52b69a7616bc490ce1358f  2026-10-28/19/2026-10-28T19.opening_state.parquet
e8d2005960af1664125cda151dab532071341380085857ac14348a0f4b7ce817  2026-10-28/19/2026-10-28T19.verify_report.json
565408551304bc641420f45000a38e37337eed95a084cde0ae548a0a77e808df  2026-10-28/19/2026-10-28T19.manifest.json

2026-10-29T00

# v3/SHA256SUMS.txt
50502b4a2735124656d68e085f1eb26dfc0b4c9f3b4ac79ba9b40c9741ed9b4c  2026-10-29/00/2026-10-29T00.parquet
91166eded9200fb2daa2b22a8b59293196d05b911216af450739d3581546ea74  2026-10-29/00/2026-10-29T00.witness_stats.json
adb353bc6d9f010902c0270ee9c54b1214d493cd011ce21a35631ff52cac3a56  2026-10-29/00/manifest.json
# v3.1/SHA256SUMS.txt (manifest, verify report, opening state; the manifest lists the other files)
1e79ebe0f212aa809ef0ae4bfec3a92147a33e7a5e801e62ebd66ad0ab3e7508  2026-10-29/00/2026-10-29T00.opening_state.parquet
34d7bb237998611d52cc7824a3d85818daefb2799ccefa6bbf305dc44d25309e  2026-10-29/00/2026-10-29T00.verify_report.json
4b9cba9008a3ff666fabe3081536be52563842a0659ad97771e4c69d2f7097f2  2026-10-29/00/2026-10-29T00.manifest.json

2026-10-29T18

# v3/SHA256SUMS.txt
c83c8fad58b0d2cb38df97fbf06456a2e9d8364ded7cdfbef01aa71407f86188  2026-10-29/18/2026-10-29T18.parquet
6f117a1ec811aeb7e9dc18589e54d7451011e1df42dad558fff94000765227a5  2026-10-29/18/2026-10-29T18.witness_stats.json
da3fb36357c72cbe6d51985e64a28440b22347047da4d2564d09389a198fa5ba  2026-10-29/18/manifest.json
a05762deaef61c0c470aaab5fdaa6dc69bef0fa9c35382efb1161e51574d9cf0  2026-10-29/18/2026-10-29T18.witness_delivered.json
# v3.1/SHA256SUMS.txt (manifest, verify report, opening state; the manifest lists the other files)
7047b8b1f1f31747c3b1b50f875a7a15a02d99ebad7bacaad381d4144673580e  2026-10-29/18/2026-10-29T18.opening_state.parquet
f633118cbc1abff1493284a6f4c3f39d020bbc5d593293e7e4349edfe86000de  2026-10-29/18/2026-10-29T18.verify_report.json
bc05ce9b0ab97dd4c07ea722975db09c92e43f28d90a78447185d8c23d6e8d46  2026-10-29/18/2026-10-29T18.manifest.json

Credit the collector: pendulumflow for V3 and V3.1, PMXT for V1 and V2, AG6 for their V2 archive. Ours is CC BY-NC 4.0, commercial licence by contact; PMXT's is CC BY 4.0; AG6 states no licence. How to cite.

Serving these bytes is not endorsing them. We are not affiliated with Polymarket.

For AI readers: llms.txt, what this archive holds and the questions it cannot answer.

https://x.com/PendulumFlow

Join the Pendulum Flow Discord: other people who build on prediction-market data, comparing notes and sharing tooling and findings.