How to read and Syncing CRM Data with the HubSpot API

If there is a starting point for nearly every HubSpot integration, it is reading CRM data. Before automating anything, before writing a single record, you need to read: take contacts into a data warehouse, mirror deals in an internal system, trigger a process from a stage change. It all starts with reading well.

And reading well in HubSpot is more than firing a GET. It is choosing the right read method for each case, understanding the object's schema before mapping fields, and keeping everything in sync over time without missing or duplicating records. In this guide, I will take you through the complete path: reading records, searching for slices, understanding properties, syncing incrementally, and handling the associations that connect everything.

Reading records of an object

The most basic case is listing records of an object, whether standard, like contacts and deals, or custom. The golden rule here is to request only the properties you will actually use, with the properties parameter, to reduce the response size and the pressure on rate limits. The limit is 100 records per page, and pagination is by cursor, so you repeat the call with the cursor until you have walked through everything.

Searching for a subset with the Search API

When you do not want everything, but a slice, by date, by stage, by value, the right tool is the Search API. It accepts grouped filters, sorting, and property selection, and returns only what matches your condition. The central caution is the cap of 10,000 results per filter set: if your slice is larger than that, break it into non-overlapping ranges, usually by creation date windows, and paginate each range separately. Using search when you should list, or vice versa, is one of the most common causes of incomplete extractions.

Understand the schema before mapping fields

A step many people skip and later regret is reading the object's properties before syncing. Before deciding which fields to take to the other system, query the schema. The response carries, for each property, the internal name, the type, the group it belongs to, and whether it is read-only. That last point is crucial: calculated fields, the so-called rollups, cannot be filled via API. Trying to write to them only generates errors. Knowing the schema beforehand keeps you from mapping a field you could never write to.

Incremental sync: only what changed

Keeping the destination current does not mean extracting the whole base every time, which would be slow and expensive. It means bringing only what changed since the last sync. The pattern is to search by hs_lastmodifieddate greater than the moment of the last run, with an overlap window of a few minutes so you do not miss records changed right at the cutoff boundary, because the Search API reads from an eventually-consistent index and a just-changed record can take moments, sometimes minutes, to appear, so the overlap should cover that lag. You paginate the search by cursor and deduplicate by id on write, so the overlap updates instead of duplicating. This incremental pattern is what turns an extraction of hours into a routine of minutes. One caveat for contacts: they carry two modified-date fields, lastmodifieddate and hs_lastmodifieddate, so choose consciously, since hs_lastmodifieddate also moves on engagement activity like email opens.

Companies, deals, and the role of associations

Companies and deals follow exactly the same read pattern as contacts: list, search, properties, incremental sync. What changes, and what makes the CRM truly a CRM, is the association. A deal does not live alone, it is linked to contacts and to a company. A contact belongs to a company. These connections are first-class data, and you need to read them to rebuild the web of relationships in the other system.

Associations paginate too

The detail that catches many people is that the list of records associated with a record is also paginated. An anchor company can have dozens of contacts linked to it, spread across several pages. If you read only the first page of associations, you lose the links of the rest, exactly the most connected and important records. Handle association reads with the same cursor loop as normal reads, no exceptions, or the web of relationships arrives incomplete at the destination.

Batch read: when you already have the ids

There is a third way to read that solves a very common specific case: you already have a list of ids and want the full data of each one. For that, instead of making a hundred individual calls, use the batch read endpoint, which accepts up to 100 ids per request and returns them all at once. This drastically reduces the number of calls and, with it, the pressure on rate limits. It is the right way, for example, to enrich a list of contacts that came from another system with the current data from HubSpot.

Batch read has no cursor, because you are not discovering records, you are requesting records you already know. Pagination there is your own splitting of the id list into blocks of up to 100. The care is not to repeat the same id within a batch and to handle the response to separate the ones that came back from the ones that, by chance, no longer exist. Combined with listing and search, batch read completes the extraction toolkit.

Why this matters for your operation

Reading and syncing the CRM well is the foundation of almost everything that comes after. A reliable BI depends on the extraction into its data warehouse not losing records. An ERP integration depends on a sync that reflects every change. A nurturing cadence depends on truly knowing who is associated with what. When reading fails, the effect does not stay technical: the dashboard lies, the decision rests on stale data, the customer gets the wrong message. Investing in reading right is investing in the trust the whole operation places in the data.

In practice: the CRM mirror that did not match

An operation mirrored the HubSpot CRM into a data warehouse to feed reports, and the numbers never matched what was seen in HubSpot. On investigation, two causes appeared. The deal extraction used search and stopped at the 10,000 cap, and the association read brought only the first page, losing the links of the larger accounts, which were exactly the most relevant for the analysis.

The fix was to break the search into date ranges and paginate associations with the same cursor loop as record reads. After that, the mirror reflected HubSpot faithfully, and the reports became trustworthy again. The data was not wrong at the source; it was arriving half-complete because of two poorly built reads.

Checklist for reading and syncing the CRM

1. Does the read request only the necessary properties in properties?
2. Do you use listing for full extraction and search for slices?
3. Are large searches broken into ranges so they do not hit the 10,000 cap?
4. Did you read the object's schema and know which fields are read-only?
5. Is the sync incremental, by hs_lastmodifieddate, with overlap and id deduplication?
6. Do association reads use the same cursor loop as record reads?
7. Is there a persisted pointer with the highest hs_lastmodifieddate processed?

Frequently asked questions

How do I do the first load and then only what changed?

Initial load by listing the whole object, which has no 10,000 cap, or by Search in createdate ranges to chunk it. After that, incremental sync by hs_lastmodifieddate greater than the last cutoff, with an overlap window and id deduplication on write.

Can I write to calculated fields?

No. Rollups and calculated fields are read-only in the API. To derive values, use a workflow instead of trying to write via API.

What is the difference between reading properties and reading records?

Reading properties returns the object schema: fields, types, and groups. Reading records returns the data itself. You read the schema once to map, and the records whenever you sync.

Listing or search to extract everything?

For a complete extraction of an object, prefer listing, which has no 10,000 cap. Reserve search for filtered slices, breaking it into ranges when it exceeds the cap.

Do I need to paginate association reads?

Yes. The list of records associated with a record is also paginated. Use the same cursor loop, or you will lose links of the most connected accounts.

Does the same pattern work for custom objects?

Yes. Listing, search, reading properties, and incremental sync work the same for standard and custom objects. Only the type in the URL changes.

Need to mirror the HubSpot CRM into a data warehouse, database, or internal system that your BI reads from, current and reliable? At Insight Sales we build the read and sync routines that support reports you can trust. Talk to our team and find out how we can help.