Setting Tables and Columns to Replicate

After you connect an integration and Stitch performs a structure sync, the next thing you’ll do is select tables and columns to replicate. In this guide, we’ll cover:

• The integrations this guide is applicable to

• How object selection works

• What’s required to select and replicate objects in Stitch

• How to set objects to replicate

• How to troubleshoot common object selection issues

• What’s supported for each integration

Applicable integrations

This guide is applicable to all database and SaaS integrations.

Note: Webhook integrations don’t support table and column selection. By default, all data sent to Stitch in a webhook message will be loaded to your destination. Refer to the webhook provider’s documentation for info about possible attributes.

How object selection works

• How objects are discovered

• Levels of object selection support

How objects are discovered

During the Discovery phase of Extraction, Stitch detects the objects available for replication in a data source. Sometimes called a structure sync, this process is how Stitch determines the structure of the tables and columns in an integration. Objects must meet the selection requirements to display and be selectable in Stitch.

In an integration’s Tables to Replicate tab, you can select the individual databases (if applicable), tables, and columns you want to replicate. Integrations will have a status of Not Configured until at least one table and column are selected.

Levels of object selection support

While the majority of Stitch’s integrations support table and column selection, some may only support table selection, while others don’t have any object selection support.

When an integration doesn’t support table and/or column selection:

• Table selection: If an integration doesn’t support table selection, all available tables and columns are automatically set to replicate. For detailed info on the data Stitch replicates from SaaS integrations, check out the Schema section of any integration’s guide.

• Column selection: If an integration supports table selection but not column selection, all available columns in selected tables will automatically be set to replicate.

Refer to the Object selection support reference for more info about what each of Stitch’s integrations support.

Object selection and replication requirements

• Requirements to select an object

• Requirements to successfully replicate an object

Requirements to select an object

For an object to be selectable in Stitch, all of the following must be true:

• The user authorizing the integration has permission to access the object in the source
• Stitch successfully completes Discovery, or a structure sync. Stitch will automatically detect new objects, but a structure sync must complete before the object will display in Stitch.
• For tables, there are additional requirements:
  • For SaaS integrations, the integration supports the API endpoint associated with the table. Refer to the integration’s schema documentation for a list of available objects and endpoints associated with them.
• For columns, there are additional requirements:
  • The column contains data typed using a supported data type. Columns containing unsupported data types will display as Not supported in Stitch.
  • For SaaS integrations backed by Singer taps, the tap must either dynamically include the field or the JSON schema used to create the given table must contain the column. Refer to the integration’s schema documentation for more info.

Requirements to successfully replicate an object

For an object to replicate successfully, the following requirements, along with the selection requirements, must be met:

• The object is set to replicate

• All parent objects are set to replicate. This means that an object that contains other objects must be set to replicate for the objects they contain to replicate.

  For example: For a column to replicate, the table that contains it must also be selected. If applicable, so must the schema containing the table, and the database containing the schema. You’ll receive an error in Stitch if the parent object isn’t selected.

• The user authorizing the destination has the required permissions for your destination type
• The name of the object adheres to your destination’s requirements. Refer to the documentation for your destination type for more info.
• For tables, there are additional requirements:
  • The table and at least one column are set to replicate
  • The table has a defined Replication Method and Replication Key, if applicable
  • For integrations that replicate files, the file meets the integration’s requirements. For example: A Google Sheet must contain a header row and a second row of data.
  • The table doesn’t exceed the maximum columns per table allowed by your destination type, if applicable
• For columns, there are additional requirements:
  • The column contains at least one non-NULL value

Setting objects to replicate

In this section:

• Selecting individual objects

• Selecting all objects

Selecting individual objects

• Tables and columns

• New and additional columns in already-selected tables

• Database views

Selecting tables and columns

To set objects to replicate:

1. Click into the integration from the Stitch Dashboard page.
2. Click the Tables to Replicate tab.
3. Locate the table or column you want to replicate. Note: If there’s a parent object such as a schema, verify that it’s also selected.

4. Click the checkbox next to the object’s name. A blue checkmark means the object is set to replicate:

   

   When a table is selected, keep in mind that:

   • For database integrations, all columns are automatically set to replicate. Note: The process varies slightly for Amazon DynamoDB and MongoDB-backed integrations, which use projection expressions and projection queries to select columns.

   • For SaaS integrations that support column selection, you need to select columns by clicking into the table. In the list of columns, click the checkbox next to the column you want to replicate.

5. If setting a table to replicate, define the table’s Replication Method, if prompted. Only integrations that support configuring Replication Methods will display this prompt.
6. Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Selecting new and additional columns in already-selected tables

What happens when you’ve added a brand-new column in the data source or you want to replicate additional columns on an already-replicating table?

For more info check out the Setting New and Additional Columns to Replicate guide.

Selecting database views

While the steps for replicating a database view are almost the same as those for replicating a table, there are some slight differences.

For more info, check out the Replicating Database Views guide.

Selecting all objects

• How the Select all feature works

• Requirements to use the Select all feature

• Selecting all tables and columns

How the Select all feature works

The Select all feature, located in an integration’s Tables to Replicate tab, allows you to select all of an integration’s tables and columns with just a few clicks.

When used to select all tables, the following occurs:

• All tables and supported columns are set to replicate. Note: Database views are not supported with this feature and won’t be selected.
• Previous column selections are overwritten. Note: Selections aren’t final until Finalize Your Selections is clicked. Clicking Cancel will restore your previous selections.
• Tables with defined Replication Methods will continue to use that Replication Method.

Requirements to use the Select all feature

To use the Select all feature, all of the following must be true for an integration:

• The integration is backed by a Singer tap
• The integration supports table selection. If the integration doesn’t support table selection, all available tables and columns will be set to replicate by default.
• For database integrations, there are additional requirements:
  • Stitch supports Log-based Incremental Replication for the database. Some database cloud providers may not support the features Stitch requires to perform Log-based Incremental Replication, even if the source database supports those features. For example: Heroku doesn’t currently support logical replication for PostgreSQL.
  • The Default Replication Method setting is available and enabled for the integration. In addition to having Log-based Incremental Replication support in Stitch, the Default Replication Method setting must be available and enabled.
• For SaaS integrations, there is an additional requirement:
  • The API the integration uses must allow the selection of all fields. Some sources - like Google Ads and Microsoft Advertising - have compatibility rules that determine the combinations of fields that can be selected together. Selecting all columns for these integrations would result in incompatible combinations and unsuccessful replication.

Refer to the Object selection support reference for a list of integrations that support this feature.

Selecting all tables and columns

1. Click into the integration from the Stitch Dashboard page.
2. Click the Tables to Replicate tab.
3. In the list of tables, click the box next to the Table Names column.

4. In the menu that displays, click Track All Tables and Fields:

   

5. Click the Finalize Your Selections button at the bottom of the page to save your data selections.

Troubleshooting common issues

In this section:

• Missing objects

• Not supported table/column messages

• Parent object not set to replicate

Missing objects

Stitch cannot detect any objects (databases, tables, etc.) at or below this level.

If you receive this message or you can’t find an object (database, table, column, etc.), it’s typically a permissions issue.

Not supported table/column messages

If you see a status of Not Supported for a table or column in a database integration, there a few potential root causes:

• Insufficient permissions. Verify that the Stitch user has all the required permissions as outlined in the Setup instructions for the database. Refer to the documentation for your database for more info.

  After you grant the required permissions and a full replication cycle has completed, the table’s Sync Status should change to Supported and be available for syncing.

• Unsupported column. If a column is displayed as Not Supported, it may be that the column contains an unsupported data type.

Parent object not set to replicate

The table containing these columns is not set to sync. Please sync the parent table to ensure replication of columns selected on this screen.

If you receive this message, it means that the object above it isn’t set to replicate. For example: The table that contains a selected column isn’t set to replicate. Parent objects must be selected for the objects they contain to also replicate.

Object selection support reference

This section details the level of support each of Stitch’s database and SaaS integrations have for the features outlined in this guide:

• Table selection: Support for setting individual tables to replicate
• Column selection: Support for setting individual columns in a table to replicate
• Select all: Support for the Select all feature

Icons in the table represent the following:

• indicates that the feature is supported
• indicates that the feature is supported, but has specific configuration requirements. Hover over the icon to view a tooltip with additional details.
• indicates that the feature isn’t currently supported. Hover over the icon to view a tooltip with additional details.

Back to Object selection support reference

Did this article help? If you have questions or feedback, feel free to submit a pull request with your suggestions, open an issue on GitHub, or reach out to us.