Paginating the HubSpot API: The Guide to Not Missing (or Duplicating) Records

Let me guess your problem. Your HubSpot integration works, runs without throwing an error, but in the end brings fewer records than it should. Or brings some duplicates that mess up the destination. Or always stops at a round, suspicious number, like 9,900, as if the entire base magically fit within that limit. If you recognized yourself in any of these symptoms, you can relax, because the culprit is almost always the same, and it has a fix: pagination.

The HubSpot API returns data in pages, and your code needs to walk through all of them, in the right order, without skipping or repeating any. It sounds like a trivial loop, but it is exactly where most of the extraction bugs we see day to day live. And they are treacherous bugs, because the integration does not break with a red error in your face. It simply delivers a wrong number, silently, and you only find out weeks later, when the pipeline report does not match what the team sees on screen.

The good news is that there is a single pagination pattern that works for almost every HubSpot endpoint. Once you understand it, you stop making mistakes. In this guide, I will show you the cursor pattern, the three ways of reading that paginate differently, the almost always forgotten detail of associations, and how to prove, at the end of the extraction, that you really covered the entire base without leaving anything behind.

The cursor pattern, which solves almost everything

Most v3 and v4 HubSpot endpoints paginate by cursor, and the mechanism is always the same. The API response carries a cursor inside paging.next.after. You take that value, repeat the exact same request passing it in the after parameter, and keep going, page after page, until a response comes back without paging.next. The disappearance of the cursor is the only reliable sign that the extraction is finished. Any other way of deciding the end, like a fixed counter, is a bet that sooner or later betrays you.

The loop is easy to describe and surprisingly easy to get wrong. You make the first call without a cursor, read the paging.next.after from the response, build the next call with that value, and repeat. The most common mistake we see is a developer storing the body or URL of the first request and resending it without updating the cursor each round. The result is an extraction stuck on the same page forever, or one that processes only the first page and thinks it is done. Always update the after with the latest value before each new call.

Why this matters for your operation

Wrong pagination is not an isolated technical detail hidden in the code with no consequence. It is a trust problem that leaks into the whole operation. When the extraction loses records, the BI shows fewer customers than really exist, the nurturing cadence leaves important contacts out, the reconciliation with another system flags differences nobody can explain. The team then spends hours chasing a ghost, distrusting the network, the destination database, the BI tool, when the root of it all is a poorly built pagination loop. Getting pagination right is getting right the invisible foundation that every report, alert, and automation will rest on later.

The three ways of reading paginate differently

There are three ways to read data in HubSpot, and each one paginates in its own way. Treating them all the same is the source of much of the bugs. The table below sums up the differences, and right after I explain each case.

Listing is the most direct of all: you paginate by cursor to the end and there is no cap on how many records it can bring in total. It is the right way for a complete extraction of an object. Search paginates exactly the same, with a cursor, but carries the cap of 10,000 records per filter set, which you work around by breaking the query into non-overlapping date ranges. Batch read has no cursor at all, because you are not discovering records, you are requesting records whose ids you already know. Pagination there is your own responsibility: splitting the id list into blocks of up to 100 and being careful not to send a repeated id in the same batch.

The almost always forgotten detail: associations paginate too

Here is a point that catches almost everyone on their first serious integration. When you read a record's associations, for example all the companies linked to a contact, or all the contacts linked to a deal, that list of associated records also comes paginated. It can also exceed one page. Many people forget this and lose exactly the links of the most connected records, which tend to be the most important in the base, like that anchor company with dozens of contacts hanging off it. The rule is simple: handle association reads with the same cursor loop you use for normal reads, no exceptions.

Why records jump pages

There is a subtle effect that confuses even experienced people. When records are created or changed during the extraction itself, they can shift position mid-way and, with that, jump an entire page or appear twice. Explicit sorting is a Search API feature, not something the plain list endpoint supports. When you extract via Search, sort by a stable, unique field such as hs_object_id ascending to anchor the sweep, because that id never changes after a record comes into existence. On the plain list endpoint, where you cannot sort, rely on the default order HubSpot returns and on the two safeguards below. This care makes little difference in a small, idle base, but it is decisive in a large, active base, where dozens of records change per minute while you extract.

Common mistakes that cost hours of debugging

• Resending the body without updating the cursor: the extraction enters an infinite loop on the same page, or processes only the first.
• Sending an empty `after` on the first call: some endpoints reject it and return an error right away.
• Stopping at a fixed counter: the total can change during the extraction and you cut the sweep too early.
• Forgetting to paginate associations: links of highly connected records disappear without warning.
• Relying on a mutable sort key in Search: in active bases, records jump pages and cause loss or duplication; use a stable, unique key like hs_object_id.
• Not validating the total at the end: the gap goes unnoticed for weeks, until someone notices the number does not add up.

How to make sure you did not miss or duplicate

Both loss and duplication appear when records move between pages mid-extraction. Three measures, applied together, solve the overwhelming majority of cases and give you the confidence to trust the result.

1. For Search extractions, sort by a stable, unique key, like hs_object_id, so records do not jump pages during the sweep, even while the base changes. The plain list endpoint cannot sort, so there you lean on the total check and id deduplication below.
2. Validate the total at the end against the object count. If the number matches, you covered everything. If it does not, the extraction has a gap that needs investigating before you trust the data.
3. Deduplicate by id when writing to the destination. This is especially important in incremental syncs with an overlap window, where bringing some duplicates is expected and even desired, and the id is what turns the duplicate into an update, not a new record.

In practice: the extraction that stopped at 9,900

An operation extracted contacts into an internal system and the number never matched what HubSpot showed. The cause was two faults combined, which made the diagnosis even more confusing. The search used hit the 10,000 cap per filter set, and at the same time the loop was not passing the cursor correctly, which cut the extraction even earlier, near 9,900 records. The team suspected the CRM, the network, the destination database, everything except pagination, which seemed too simple to be the problem.

The fix had two steps. First, repair the cursor loop to actually update the after each round. Second, break the search into creation date ranges so none of them hit the cap. After that, the total matched the object count exactly. And, more importantly, validating the total became a permanent part of the routine, so any future gap shows up immediately, not weeks later in a forecast meeting. Trust in the number came back, and with it trust in everything that depended on that base.

Checklist of a failproof pagination

1. Do you omit after on the first call and only start sending it when there is a cursor?
2. Does the loop read the after from the response and inject the updated value into the next request?
3. Does the loop stop when paging.next disappears, and not at a fixed counter?
4. Are large searches broken into ranges so they do not hit the 10,000 cap?
5. Do association reads use the same cursor loop as normal reads?
6. In Search sweeps, is the sort key stable and unique, such as hs_object_id, during the sweep?
7. Do you validate the extracted total against the object count at the end of each run?

Frequently asked questions

Why do I get an error sending after on the first call?

Some endpoints reject an empty or malformed after. Always omit the parameter on the first page and only add it when the response carries a real cursor.

What is the record limit per page?

100 on most listing and batch endpoints. Asking for more is rejected by the API, so the path for large volumes is to paginate, not raise the limit.

How do I know pagination has finished?

When the response no longer carries paging.next.after. That is the end-of-loop signal. Never stop at a fixed counter, because the total can change during the extraction.

Do I need to paginate association reads?

Yes. The list of records associated with a record can also exceed one page. Use the same cursor loop, or you will lose links of highly connected records.

How do I avoid bringing the same record twice?

In a Search sweep, sort by a stable, unique key like hs_object_id, validate the total at the end, and deduplicate by id when writing, especially in syncs with an overlap window.

Listing or search to extract everything?

For a complete extraction of an object, prefer listing, which has no 10,000 cap. Reserve search for filtered slices, and break it into ranges when the slice exceeds the cap.

Want extraction routines that run start to finish without gaps or duplicates? At Insight Sales we build HubSpot collections and data flows ready to operate at volume. Talk to our team and find out how we can help.