Netlify Docs
Ask Netlify

Contact

Manage data /Connect /

Sync events in Netlify Connect

This feature is available on Connect Enterprise plans.

Netlify Connect automatically syncs and updates your data layer to ensure that you always have access to the latest data.

This document covers the types of sync events that occur, what triggers automatic data syncs, and how to trigger a sync manually in the Netlify UI or through webhooks.

To learn how to access sync event logs and set up Slack notifications for failures, review monitor activity.

# Sync event types

There are two types of events that can occur in Connect:

  • Sync from all data sources: the system reads data source configurations, installs dependencies, and then retrieves data from all data sources. Once the data is retrieved, the system uses the information to build or update the GraphQL schema for the data layer
  • Sync from {data source type}: the system retrieves data from the specified data source

Netlify does a full sync to set up or update your data layer when you create a new data layer and when you add or update data source configurations, cross-references, and API scopes. Netlify also does a full sync when you manually trigger a sync. When a full sync occurs, the event is Sync from all data sources.

When there is a data update, such as newly published content in your CMS, Netlify syncs the updated data source. If the webhook notification from the data source includes a header to specify that only that data source should update, the event is Sync from {data source type}. If not, the update will trigger a full sync and the event will be Sync from all data sources.

Each successful sync generates a new data revision. You can review all available data revisions in the Revisions section on the data layer’s overview page. Learn more about data revisions.

# Automated syncs

One of the benefits of Connect is that it automatically updates your data layers to ensure they always have access to the latest data.

The API will use the pinned revision, even if syncs generate new ones

If you pin a data revision, successful syncs will continue to generate new revisions but the GraphQL API for your data layer will always use the pinned one.

Automated syncs occur in the following scenarios:

  • When you create a new data layer and when you add or update data source configurations, cross-references, and API scopes. This triggers a sync of all data sources and Netlify creates a new data revision and updates the GraphQL schema and API to reflect the change.
  • When data in a data source updates, such as when you publish new content in your CMS. Netlify is notified of the change and automatically creates a new data revision and updates your data layer.

# Manual syncs

If you are troubleshooting and want to manually trigger a refresh of all data in your data layer, you can start a sync from all data sources through the Netlify UI or a webhook.

Note that if your data layer’s API is using a pinned data revision, Netlify will generate a new revision to reflect the data sync but the API will continue to use the old data. Make sure to unpin the old data revision and/or pin the new revision once the data refresh is complete.

# Trigger a sync with the Netlify UI

  1. Navigate to the

    Connect
    page for your team and select the data layer from the Data layers list.

  2. In the Activity section, select

    Trigger sync Sync from all data sources
    . Netlify will start syncing the data.

# Trigger a sync with the webhook

You can use webhooks in external systems to trigger a sync in Connect.

Keep in mind the following before you set up your webhook:

  • Webhooks should act only as notifications to let Connect know that content updates are available and that Connect should start a new sync to get the latest data.
  • Webhooks should be stateless and they should not contain information about what needs to be updated.
  • If multiple webhook notifications come in at the same time, the newest one will take precedence and all others will be dropped.
  • If a new webhook notification comes in while a sync is in process, Connect will process that webhook immediately after the current sync finishes.

To set up a webhook to trigger a sync:

  1. Navigate to the
    Connect
    page for your team and select the data layer from the Data layers list.
  2. Select Data layer settings.
  3. Find the Webhook URL under
    General Data layer information
    . It is formatted as https://webhook.netlify-connect.com/hooks/data_layer/data_source/publish/{data_layer_id}.
  4. If you wish to only update a single data source or add a webhook body, add a x-connect-data-source header to the request with the data integration’s slug as the header’s value.
  5. Send a POST request to this webhook and Netlify will start syncing the data.

# Review data layer sync events

Netlify retains sync event logs for five days. Review the monitor activity doc to learn how to access logs for data layer sync events and how to set up Slack notifications for sync event failures.

← Monitor activity →