BigQuery
Connect BigQuery with a service account key, choose the billing project and dataset, and set a scan limit to keep query costs under control.
BigQuery needs the Starter plan or above. DataxLand runs read queries in the project you specify (Google bills the scanned bytes to that project) and copies the results into the workspace's own warehouse.
1. Prepare Google Cloud
- Create a dedicated service account for DataxLand.
- Grant BigQuery Job User on the project that runs the queries and BigQuery Data Viewer on the dataset to read (if the data lives in the same project, grant both there).
- Create a JSON key for the service account and download the file.
2. Connect
- Data sources → Add source, pick BigQuery.
- Project that runs queries (billed): the project charged for scanned bytes.
- Dataset:
datasetorproject.dataset(add.tableto read a single table). - Scan size limit: the maximum bytes a single query may scan, 2 GB by default. A query over the limit is stopped before it runs, with a message showing the estimated size.
- Service account key: paste the JSON file's contents, or use Or choose a JSON file.
- Click Test connection, pick tables under Choose the tables to read (optionally with Filter by date (optional)) and add the source.
The key is encrypted at rest and never shown again (the source card shows Password saved).
3. Sync
BigQuery supports Full and Incremental. Sync large tables incrementally on an updated-at column to cut the bytes scanned. See Sync modes and refresh schedules.
Si investigates with SQL
Si may run read-only SQL on BigQuery while investigating, within the same scan limit; a query over the limit waits for your confirmation.
Related
SQL Server / Azure SQL
Connect Microsoft SQL Server, Azure SQL Database, Azure SQL Managed Instance or Amazon RDS for SQL Server with a read-only user, and load tables in full or incrementally by rowversion or an update column.
Custom REST API
Connect any software with a REST API that returns JSON (POS, CRM, inventory, in-house ERP) by declaring the base URL, authentication, endpoints and pagination, then test the call before adding it.