Files and storage
Connect S3 to DataxLand
The S3 connector reads CSV, Parquet, Avro or JSONL files in an Amazon S3 bucket (or an S3-compatible store with a public address). You declare each stream with a path pattern, for example every orders/*.csv file, and each stream becomes a table in the workspace warehouse. It suits sales software or ERPs that export files to S3 on a schedule - Si reads a masked data sample to propose a mapping to the standard model, and you get net revenue, cost of goods sold and contribution margin just as with an Excel upload, but refreshed automatically on a schedule.
Free plan with no time limit · No card required · Sample workspace ready in minutes
What you can do with S3
- Refresh your profit and loss dashboard automatically from the daily exports of your sales software.
- Combine order files in S3 with marketplace and ad data.
- Read Parquet files exported by internal systems without opening up a database.
What data you get
Each stream lands as its own table in the workspace warehouse, as is; personal-data columns are hashed on load.
| Stream (table) | Contents |
|---|---|
| (stream you name) | Each stream has a name, a path pattern (glob) and a format; all files matching the pattern are combined into one table. |
| CSV | Options for delimiter, character encoding and header row. |
| Parquet, Avro | Read using the schema stored in the file. |
| JSONL | One JSON record per line; nested fields become JSON strings. |
Connection details
| Sign-in method | Connection details (API key, account), encrypted at rest |
|---|---|
| Sync modes | Full (reloaded each run) · Incremental (new or changed rows only) |
| Fields to fill in | Bucket, Authentication, Streams |
| Developer | Airbyte (certified), run inside DataxLand |
| License | ELv2 |
Passwords and keys are encrypted at rest. Personal-data columns are hashed on load.
Metrics computed from this source
Real definitions from the analytics packs; dashboards and Si share them.
| Metric | Definition | Unit |
|---|---|---|
| Net revenue | Line value after line discounts and allocated order vouchers, minus refunds to buyers. Refunds are deducted only here. | Currency |
| Cost of goods sold | Quantity times unit cost on the order date (half-open validity interval). SKUs without a cost count as 0; see COGS coverage. Returned lines do not reverse their COGS. | Currency |
| Contribution margin | Net revenue minus COGS, marketplace fees, payment fees, shipping and return costs (before advertising).= Net revenue - Cost of goods sold - Marketplace fees - Payment fees - Seller-paid shipping cost - Return cost | Currency |
| Valid orders | Orders that were not cancelled (standard status other than cancelled). | Count |
Connect S3 in 4 steps
- 1
Create read access
Create an IAM user (or role) that can only list and read the bucket you need, and get the access key ID and secret access key.
- 2
Add the S3 source
In the workspace choose Add source, find S3, and enter the Bucket, the IAM User (or IAM Role) authentication, and the AWS region if needed.
- 3
Define streams
Add each stream with a name, a path pattern such as orders/*.csv and the file format, then click Test connection.
- 4
Review and build
Si proposes a mapping for each stream to the standard model; you approve, then build.
Limits and caveats
- The metrics on this page are examples for files that contain orders and cost prices; the actual metrics depend on the files you provide.
- Loads are incremental by file modification time, so each refresh reads only new or recently modified files. If new files repeat old rows, you need a primary key to merge them without duplicates.
- S3-compatible stores (MinIO, R2...) need a public address; DataxLand blocks private network addresses.
- Requires the Starter plan or above. An Airbyte connector (ELv2) run inside DataxLand.
Frequently asked questions
Can it read Excel files in S3?
This page covers CSV, Parquet, Avro and JSONL only; for .xlsx files, use DataxLand's Excel source or export to CSV.
Are access keys stored safely?
Access keys are encrypted when saved and never shown again; use a user that can only read that one bucket.
What about many files with the same structure?
Declare one stream with a path pattern that matches all of them; they are combined into one table.
Dashboard templates that use this source
Connect S3 and get verified dashboards
Free plan with no time limit · No card required · Sample workspace ready in minutes