Two awkward shapes, and the cloud in the middle
Machine learning and data teams move training sets to an annotation vendor and get labels back, ship datasets from a lab to the cloud where the GPUs are, and pull checkpoints back down for evaluation. Small samples make per-file overhead dominate; sharded archives mean one stalled stream wastes a night.
Consumer sharing tools cap sizes or meter the bytes, sync tools choke on millions of files, and a browser upload over TCP to another continent never reaches the line rate you pay for.
Small files batched, big shards in parallel
Boita moves data with an accelerated transfer engine over UDP with its own rate control, holding your line rate whatever the distance. A tree of millions of small files is grouped into transfers by the desktop app or the agent, in batches of up to 500 files or a ten-second window, so the per-file cost of an ordinary upload does not apply. The desktop app resumes by itself and skips completed files.
The server records a checksum for every file, and the delivery receipt carries per-file checksums with a CSV verification report, so an evaluation run uses the checkpoint that was actually written.
One rule decides whether this helps you: sessions are per file. Four shards move on four sessions; one shard moves on one, at about 45 Mbps, the same on every plan. A plan buys sessions, not a faster single file; sharding a checkpoint turns them into throughput.
Fits training scripts and object storage
The boita command line runs in a training script or a scheduler job on the Linux agent, and the REST API with a Python SDK, scoped API keys and signed webhooks (Pro and above) does the same from your own code. Import from an S3-compatible, Google Cloud Storage or Azure Blob bucket pulls a dataset into the workspace on our servers, not through a laptop, and a sync rule on any plan copies results back. On Enterprise, auto-delivery also pushes to your own S3-compatible bucket, sub-folders kept and retried on failure.
An annotation vendor sends labels back through an upload portal with a form for batch identifiers and file-type rules, into its own dated inbox folder, with an email to both sides when it lands.
A worked example
A 1 TB dataset leaves a lab on Pro, over a 1 Gbps line, in about five hours, plan-limited at 450 Mbps; a 40 GB checkpoint on the same plan takes about twelve minutes. The same 1 TB on Scale over a 1 Gbps line is line-limited and takes about two and a quarter hours. What the far end sees depends on the distance and loss on its side; the calculator on the home page models it.
- Millions of small files batched into transfers; huge shards in parallel sessions
- boita CLI, REST API and Python SDK for scripts (Pro and above)
- Import from and sync with S3-compatible, Google Cloud Storage or Azure Blob buckets
- Upload portals with forms for annotation vendors
- Checksum on every file; receipts with a CSV verification report
Controls on proprietary data
Sender-side encrypted delivery with a passphrase keeps a dataset unreadable to anyone, including Boita, until the recipient enters it. Share links carry an expiry, an optional password and a download limit, recipient verification by email code names who downloaded, and a revoked link stops resumed downloads within minutes. Folder permissions per member or group (Scale and above) give a vendor’s contact one folder and nothing else. Two-factor authentication and an audit log are on every plan; SSO, IP allow-lists and audit export to a SIEM are on Enterprise.