Sync modes and refresh schedule
The three ways to sync data — full, incremental and CDC (change log) — which sources support which, and how to set an automatic refresh schedule.
Three sync modes
| Mode | How it works | Keep in mind |
|---|---|---|
| Full | Re-reads the whole table on every refresh | Simplest; slow on large tables |
| Incremental | Reads only rows whose cursor column (for example updated_at) is newer than last time, merged on the primary key | Does not see rows deleted at the source |
| CDC (log-based) | Reads the database's change log, including deleted rows | PostgreSQL only; logical replication must be enabled |
| Source | Full | Incremental | CDC |
|---|---|---|---|
| Excel files | Yes | — | — |
| PostgreSQL | Yes | Yes | Yes |
| MySQL / MariaDB, SQL Server | Yes | Yes | — |
| BigQuery, marketplaces, connectors | Yes | Yes | — |
File sources are always re-read in full. Marketplaces (Shopee, TikTok Shop, Lazada) are read incrementally by order update time.
Change the sync mode
- Go to Data sources, card Sync method, and click Edit sync method.
- Choose a mode for each table. For Incremental, pick the Cursor column (columns marked "(suggested)" are DataxLand's suggestions) and the Primary key.
- Click Save sync method. The next refresh does one full load for every table whose settings changed.
When a new workspace is created, every table is loaded in full; you can change the sync mode once the workspace exists. For CDC, read the requirements under PostgreSQL.
Refresh schedule
- On the Data refresh schedule card, choose a Frequency: every 15 minutes, every 30 minutes, hourly, daily, weekly or monthly.
- Choose the Time (for daily, weekly and monthly schedules), and the Day of week or Day of month (1–28).
- Switch on Turn on automatic refresh and click Save schedule. The default is daily at 06:00 in the workspace time zone.
Every 15 and every 30 minutes are only available when every source syncs incrementally or with CDC. Schedules only run once the workspace is approved.
| Plan | Frequencies available |
|---|---|
| Free | No schedule; upload new files by hand (Thay file, replace file) |
| Starter | Hourly, daily, weekly, monthly |
| Small Business and above | Down to every 15 minutes |
A schedule your current plan does not include is skipped, with the reason logged. When storage goes above 110% of your quota, refreshes pause; see Plans and quotas.
Related
Workspace
A workspace is the unit of data, members and plan in DataxLand — its states from creation to ready, the navigation bar and the roles.
Si: context analysis and Q&A
Si is DataxLand's AI data assistant — it proposes how to read and map your data for you to approve, and answers questions with approved metrics.