# Storage Mounts (https://tenki.cloud/docs/sandbox/storage) > For the complete documentation index, see [llms.txt](https://tenki.cloud/llms.txt) Mount S3, R2, GCS, and S3-compatible buckets as folders in a sandbox. A storage mount shows a bucket, or one prefix inside it, as a folder in a sandbox. Programs in the sandbox read and write the objects as ordinary files. Tenki runs the mount on the sandbox's host, so bucket credentials never enter the sandbox. Tenki gives a mount's credentials only to the host that runs the sandbox, and stops once the sandbox is paused or terminated. The mount also keeps working when the sandbox has no outbound network access. With access keys, the host uses the key itself, so a mount's prefix and read-only setting limit what the sandbox can do, not what the key allows. Give each connection a key limited to the buckets and actions its mounts need. Storage mounts work with Amazon S3, Cloudflare R2, Google Cloud Storage, and S3-compatible services such as MinIO, Wasabi, Backblaze B2, and Tigris. Storage mounts are in preview and are enabled per workspace. Contact Tenki support to turn them on for yours. The **Storage** tab under **Settings** appears once they are on. Use a storage mount for data that already lives in a bucket or that other tools read, such as datasets, model weights, build artifacts, and logs. For caches and other data that only sandboxes use, a [persistent volume](https://tenki.cloud/docs/sandbox/volumes.md) is faster and behaves like a local disk. Setting up a bucket takes three steps: 1. [Add a storage connection](#add-a-connection) for the bucket, once per workspace. 2. [Test the connection](#test-a-connection). 3. Mount it as a [default mount](#default-mounts) for every new sandbox, [when you create a sandbox](#mount-at-create), or [on a running sandbox](#attach-and-detach). The CLI commands are under `tenki sandbox storage` and need Tenki CLI **2.10.0 or later**. The SDK methods are in **1.4.3 or later** of `@tenkicloud/sandbox` for TypeScript and `tenki` for Python, and **1.4.4 or later** of `github.com/LuxorLabs/tenki-sdk-go/sandbox` for Go. ## Storage connections A storage connection records the provider, bucket, endpoint, region, default access, and how Tenki signs in to the bucket. Default access is read-only or read-write. A mount uses it unless the mount sets its own access. Tenki stores credential values as [Tenki Secrets](https://tenki.cloud/docs/secrets/manage-secrets.md) in the `s3_storage` scope, and the connection keeps only the secret names. Tenki never passes `s3_storage` secrets to CI jobs or into sandboxes. They always belong to the whole workspace and cannot be moved to a repository. Because connections write and delete secrets, adding, updating, or deleting one needs the same access as the Secrets API. Your account needs workspace-edit permission, and an API key or CLI login needs [**Allow Secrets access**](https://tenki.cloud/docs/secrets/manage-secrets.md#access-permissions). Listing and testing connections, editing default mounts, and mounting buckets do not need Secrets access. | Provider | Authentication | You provide | | -------------------- | ------------------------------------------------------ | -------------------------------------------------------------------- | | Amazon S3 | [Keyless IAM role](#keyless-aws), recommended | Bucket, region, and the ARN of an IAM role in your own AWS account | | Amazon S3 | [Access keys](#aws-access-keys) | Bucket, region, and an access key pair | | Cloudflare R2 | [Access keys or scoped temporary keys](#cloudflare-r2) | Bucket, account ID, and R2 API token credentials | | Google Cloud Storage | [HMAC keys](#google-cloud-storage) | Bucket and the HMAC key of a service account | | S3-compatible | [Access keys](#s3-compatible) | Bucket, endpoint, region if the service needs one, and an access key | ### Add a connection In the dashboard: 1. Open **Settings > Storage** and click **Add connection**. 2. Enter a name, pick the default access and the provider, and enter the bucket and the provider's fields. 3. Pick the authentication method and fill in its fields. For each credential, choose **Enter value** to store a new secret, or **Use secret** to pick an existing `s3_storage` secret. 4. Click **Add connection**. With the CLI, credential values come from environment variables or files, never from command arguments. Each credential has three flags, and you pass one of them: * `---env NAME` stores the value of the environment variable `NAME` as a new secret. * `---file PATH` stores the contents of a file as a new secret. * `---secret NAME` uses an existing `s3_storage` secret. The fields are `access-key-id`, `secret-access-key`, `session-token`, and `r2-parent-token`. This example adds a read-only connection to an R2 bucket with access keys: **TypeScript** ```typescript import { TenkiSandbox } from "@tenkicloud/sandbox"; const sandbox = new TenkiSandbox(); const connection = await sandbox.createStorageConnection({ name: "media", provider: "R2", endpoint: "https://.r2.cloudflarestorage.com", bucket: "media", authMethod: "ACCESS_KEY", defaultAccess: "READ_ONLY", credentials: { accessKeyId: { value: process.env.R2_ACCESS_KEY_ID! }, secretAccessKey: { value: process.env.R2_SECRET_ACCESS_KEY! }, }, }); ``` To use an existing secret, pass `{ secretName: "MY_SECRET" }` instead of `{ value }`. **Python** ```python import os from tenki import Client, StorageConnectionCredentials, StorageSecret client = Client() connection = client.storage_connections.create( name="media", provider="R2", endpoint="https://.r2.cloudflarestorage.com", bucket="media", auth_method="ACCESS_KEY", default_access="READ_ONLY", credentials=StorageConnectionCredentials( access_key_id=StorageSecret.from_value(os.environ["R2_ACCESS_KEY_ID"]), secret_access_key=StorageSecret.from_value(os.environ["R2_SECRET_ACCESS_KEY"]), ), ) ``` To use an existing secret, pass `StorageSecret.named("MY_SECRET")` instead. **Go** ```go connection, err := client.CreateStorageConnection(ctx, tenkisandbox.CreateStorageConnectionOptions{ Name: "media", Provider: tenkisandbox.StorageProviderR2, Endpoint: "https://.r2.cloudflarestorage.com", Bucket: "media", AuthMethod: tenkisandbox.StorageAuthMethodAccessKey, DefaultAccess: tenkisandbox.StorageAccessReadOnly, Credentials: tenkisandbox.StorageConnectionCredentials{ AccessKeyID: tenkisandbox.StorageSecretValue(os.Getenv("R2_ACCESS_KEY_ID")), SecretAccessKey: tenkisandbox.StorageSecretValue(os.Getenv("R2_SECRET_ACCESS_KEY")), }, }) ``` To use an existing secret, pass `tenkisandbox.StorageSecretName("MY_SECRET")` instead. **CLI** ```bash tenki sandbox storage connections create media \ --provider r2 \ --bucket media \ --s3-endpoint https://.r2.cloudflarestorage.com \ --default-access ro \ --access-key-id-env R2_ACCESS_KEY_ID \ --secret-access-key-file ~/r2-secret-access-key ``` Tenki names the new secrets after the connection, for example `STORAGE_MEDIA_ACCESS_KEY_ID` and `STORAGE_MEDIA_SECRET_ACCESS_KEY`, and records which secrets it created for the connection. Entering a value never overwrites a secret Tenki did not create for this connection. List connections with `tenki sandbox storage connections list`, or with `listStorageConnections()` in TypeScript, `client.storage_connections.list()` in Python, and `ListStorageConnections` in Go. ### Keyless AWS With a keyless IAM role, Tenki stores no AWS keys. The role is an IAM role in your own AWS account, and its policies decide what Tenki can reach. For each mount, Tenki's OIDC issuer signs a short-lived token, AWS STS exchanges it for short-lived credentials, and Tenki renews them while the mount runs. Tenki also attaches a session policy to those credentials, so each mount reaches only its own bucket, prefix, and access, even when the role allows more. 1. Pick a name for the role, such as `tenki-storage`. Its ARN will be `arn:aws:iam:::role/tenki-storage`. 2. Add the connection in Tenki with provider **Amazon S3**, authentication **Keyless IAM role (recommended)**, the bucket, its region, and that role ARN. The dashboard then opens **Trust Tenki from your AWS account**, which shows Tenki's provider URL, the audience, and the trust policy for this connection. To open it again later, choose **Show trust policy** from the connection's menu. 3. In the AWS IAM console, add an OpenID Connect identity provider with Tenki's provider URL and the audience `sts.amazonaws.com`. One identity provider per AWS account serves every connection. 4. Create the role with the name you picked, choose **Custom trust policy**, and paste the trust policy Tenki showed. 5. Give the role a permissions policy for the bucket, like the one below. 6. Back in Tenki, [test the connection](#test-a-connection). With the CLI, step 2 looks like this: ```bash tenki sandbox storage connections create datasets \ --provider aws \ --region us-east-1 \ --bucket my-datasets \ --auth aws-role \ --role-arn arn:aws:iam::123456789012:role/tenki-storage \ --default-access ro tenki sandbox storage connections list --json ``` The connection's `oidc_subject_pattern` and the list's `oidc_issuer` and `oidc_audience` fill in the trust policy. Use the issuer without `https://` as ``: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::123456789012:oidc-provider/" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringEquals": { ":aud": "sts.amazonaws.com" }, "StringLike": { ":sub": "workspace::storage::session:*" } } } ] } ``` The subject names your workspace and this connection, so another workspace that learns the role ARN cannot assume the role. Keep the condition exactly as Tenki shows it. A broader `sub` pattern lets other connections, or other workspaces, use the role. This permissions policy allows read-write mounts of `my-datasets`: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": ["s3:ListBucket", "s3:GetBucketLocation"], "Resource": "arn:aws:s3:::my-datasets" }, { "Effect": "Allow", "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject", "s3:AbortMultipartUpload"], "Resource": "arn:aws:s3:::my-datasets/*" } ] } ``` Read-only mounts use only `s3:ListBucket`, `s3:GetBucketLocation`, and `s3:GetObject`, so drop the other actions if you never mount the bucket read-write. In the SDKs, set the authentication method to `AWS_OIDC_ROLE` and pass only the role ARN: `credentials: { roleArn }` in TypeScript, `StorageConnectionCredentials(role_arn=...)` in Python, and `StorageConnectionCredentials{RoleARN: ...}` in Go. ### AWS access keys Use access keys when you cannot add an identity provider to the AWS account. Create an IAM user with the permissions policy above, create an access key for it, and add the connection with authentication **Access keys**. ```bash tenki sandbox storage connections create datasets \ --provider aws \ --region us-east-1 \ --bucket my-datasets \ --default-access ro \ --access-key-id-env AWS_ACCESS_KEY_ID \ --secret-access-key-env AWS_SECRET_ACCESS_KEY ``` Tenki derives the endpoint from the region, so leave the endpoint empty. Tenki still limits each mount to its prefix and access on the host, but the key itself can do whatever its IAM policy allows, so limit that policy to the bucket. The optional session token is for temporary AWS credentials. Tenki cannot renew those, and mounts fail once they expire, so prefer a long-lived key or a keyless role. ### Cloudflare R2 An R2 endpoint has the form `https://.r2.cloudflarestorage.com`, where the account ID is the 32-character ID in the Cloudflare dashboard. For a bucket in a jurisdiction, add it to the host, as in `https://.eu.r2.cloudflarestorage.com`. The Tenki dashboard asks for the account ID and jurisdiction and builds the endpoint for you. R2 uses the region `auto`, so leave the region empty. R2 connections authenticate in one of two ways. With **Access keys**, Tenki uses the access key ID and secret access key of an R2 API token. Create the token in the Cloudflare dashboard under **R2 > Manage API tokens**, with **Object Read & Write**, or **Object Read only** for a read-only connection. The token can be limited to this bucket. With **Scoped temporary keys from an API token**, Tenki asks Cloudflare for a short-lived R2 key for each mount, limited to the mount's bucket, prefix, and access. This method needs two values: 1. The access key ID of an account-level R2 API token with **Object Read & Write**. Temporary keys are derived from it. A token limited to specific buckets does not work, because Cloudflare's temporary credentials API rejects it. 2. A Cloudflare API token with the **Workers R2 Storage Write** permission, which Tenki uses to request the temporary keys. Tenki does not need the R2 token's secret access key for this method. ```bash tenki sandbox storage connections create media \ --provider r2 \ --bucket media \ --s3-endpoint https://.r2.cloudflarestorage.com \ --default-access rw \ --auth r2-temporary \ --access-key-id-env R2_PARENT_ACCESS_KEY_ID \ --r2-parent-token-file ~/cloudflare-api-token ``` In the SDKs, the method is `R2_TEMPORARY` and the API token goes in the `r2ParentToken` credential (`r2_parent_token` in Python, `R2ParentToken` in Go). ### Google Cloud Storage Tenki reaches Google Cloud Storage through its S3-compatible XML API with an HMAC key. 1. Create a service account and grant it access to the bucket: read access for a read-only connection, or read and write access otherwise. 2. In the Google Cloud console, open **Cloud Storage > Settings > Interoperability** and create an HMAC key for that service account. Google shows the secret only once, so copy it then. 3. Add the connection with provider **Google Cloud Storage** and authentication **HMAC keys**. The key's access ID goes in **Access key ID**, and its secret in **Secret access key**. ```bash tenki sandbox storage connections create training \ --provider gcs \ --bucket my-training-data \ --default-access ro \ --access-key-id-env GCS_HMAC_ACCESS_ID \ --secret-access-key-file ~/gcs-hmac-secret ``` Tenki sets the endpoint to `https://storage.googleapis.com`, and the region can stay empty. Use a service account key rather than one tied to a person, so the connection keeps working when people leave and you can limit the account to the buckets it needs. ### S3-compatible Use provider **S3-compatible** for MinIO, Wasabi, Backblaze B2, Tigris, and other services that speak the S3 API. Enter the service's endpoint and an access key pair. Set the region when the service signs requests with one, such as `us-west-004` for a Backblaze B2 bucket in that region. Without one, the region is `auto`. ```bash tenki sandbox storage connections create backups \ --provider s3 \ --bucket my-backups \ --s3-endpoint https://s3.us-west-004.backblazeb2.com \ --region us-west-004 \ --default-access rw \ --access-key-id-env B2_KEY_ID \ --secret-access-key-env B2_APPLICATION_KEY ``` The endpoint must be a public `https://` URL with no path. Saving a connection fails for IP addresses, `localhost`, and hosts ending in `.localhost`, `.local`, or `.internal`. Tenki also refuses to connect to a host that resolves to a private, link-local, or other non-public address, and a connection test then reports `ENDPOINT_BLOCKED`. A MinIO server on a private network cannot be mounted. Mounts also cannot reach the addresses of the Tenki host they run on. ### Test a connection Testing checks that Tenki can reach the bucket with the connection's credentials. In the dashboard, choose **Test connection** from the connection's menu, and the result appears next to its name. From the CLI: ```bash tenki sandbox storage connections test media ``` The command prints `Bucket reachable`, or fails with the reason. With `--json`, it prints `reachable`, `reason`, and `message`, and exits with status 1 when the bucket is unreachable. The SDKs return the same fields from `testStorageConnection(id)` in TypeScript, `client.storage_connections.test(id)` in Python, and `TestStorageConnection(ctx, id)` in Go. | Reason | What to check | | ----------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | | `ACCESS_DENIED` | The key or role lacks permission on the bucket, or the keyless trust policy does not match this connection. | | `BUCKET_NOT_FOUND` | The bucket name, and the account or endpoint it belongs to. | | `REGION_MISMATCH` | The bucket is in a different region from the connection. | | `SECRET_UNAVAILABLE` | A secret the connection reads was revoked or deleted. Update the connection with a new value. | | `CREDENTIALS_UNAVAILABLE` | The provider did not issue temporary credentials. Check the AWS identity provider and trust policy, or the R2 tokens. | | `ENDPOINT_BLOCKED` | The endpoint resolves to an address that is not public, such as a private or link-local one. | | `DNS_FAILED`, `TLS_FAILED`, `UNREACHABLE` | The endpoint's hostname, its TLS certificate, and whether the service is up. | ### Update or delete a connection ```bash tenki sandbox storage connections update media --default-access rw tenki sandbox storage connections update media --secret-access-key-file ~/new-r2-secret tenki sandbox storage connections delete media ``` Credentials you leave out of an update keep their current secrets. Updates and deletes check the connection's revision, so a change someone else saved first makes yours fail instead of overwriting it; read the connection again and retry. Tenki rejects an endpoint or region change while running sandboxes mount the connection, because those mounts would keep using the old endpoint. The error names how many mounts use it. Detach them first, then update the connection. A connection cannot be deleted while a running sandbox or a default mount uses it. Deleting a connection also deletes the secrets Tenki created for it, unless another connection uses them. That includes secrets created under an earlier name of a renamed connection. Secrets you picked with **Use secret** or `---secret` stay in Tenki Secrets, even when their names look like generated ones. ### Rotate or revoke credentials To rotate a key, [rotate the secret](https://tenki.cloud/docs/secrets/manage-secrets.md#rotate-a-value) or update the connection with the new value. Running mounts pick it up at their next credential refresh, within about 15 minutes. Revoking a secret stops new mounts right away, and connection tests report `SECRET_UNAVAILABLE`. Running mounts lose access at their next credential refresh, even when programs in the sandbox have files open. They then show `failed` with `credentials_revoked`, and writes that were not uploaded yet do not reach the bucket. ### Disabled by Tenki Tenki support can disable a connection, for example after an abuse report, or detach a single mount. A disabled connection shows **Disabled by Tenki** in **Settings > Storage** and `disabled by Tenki, contact support` in `tenki sandbox storage connections list`. It cannot be edited, tested or deleted, and only Tenki support can enable it again. You also cannot create another connection to the same bucket in that workspace while it is disabled. While it is disabled: * Creating a sandbox or attaching a mount that names the connection fails with `storage connection "" was disabled by Tenki; contact support`. * Default mounts on the connection are skipped, so new sandboxes still start without them. * Tenki can detach running mounts right away, and they then show `detached_by_tenki`. Otherwise they lose access at their next credential refresh and show `failed` with `connection_disabled`. A mount Tenki detached shows `detached_by_tenki`; while Tenki finishes cutting it, it can show `unmounting` and its path stays taken. In both cases, writes that were not uploaded yet do not reach the bucket. Contact Tenki support with the workspace and connection name to find out why and to restore access. ## Mount a bucket Every mount has a connection, a mount path in the sandbox, an optional prefix, and an access mode. With a prefix, the folder shows only the objects under that prefix, and the mount cannot reach the rest of the bucket. A mount without an access mode uses the connection's default access. ### Default mounts Default mounts apply to every new sandbox in the workspace, so they suit buckets most sandboxes need, such as a shared dataset. A workspace can have up to 16 default mounts. Changing the defaults affects sandboxes created afterwards; running sandboxes keep their mounts. In the dashboard, open **Settings > Storage**, click **Edit defaults** under **Default mounts**, click **Add mount** for each mount, and click **Save defaults**. **TypeScript** ```typescript const defaults = await sandbox.getStorageDefaults(); const { connections } = await sandbox.listStorageConnections(); const datasets = connections.find((connection) => connection.name === "datasets")!; await sandbox.saveStorageDefaults( { workspaceId: defaults.workspaceId, mounts: [ ...defaults.mounts, { connectionId: datasets.id, prefix: "imagenet", mountPath: "/mnt/imagenet", access: "READ_ONLY" }, ], }, defaults.revision, ); ``` **Python** ```python from tenki import Client, StorageDefaultMount, StorageDefaults client = Client() defaults = client.storage_connections.get_defaults() datasets = next(c for c in client.storage_connections.list().connections if c.name == "datasets") client.storage_connections.save_defaults( StorageDefaults( workspace_id=defaults.workspace_id, mounts=( *defaults.mounts, StorageDefaultMount( connection_id=datasets.id, prefix="imagenet", mount_path="/mnt/imagenet", access="READ_ONLY", ), ), ), expected_revision=defaults.revision, ) ``` **Go** ```go defaults, err := client.GetStorageDefaults(ctx, "") if err != nil { log.Fatal(err) } mounts := append(defaults.Mounts, tenkisandbox.StorageDefaultMount{ ConnectionID: datasetsConnectionID, Prefix: "imagenet", MountPath: "/mnt/imagenet", Access: tenkisandbox.StorageAccessReadOnly, }) _, err = client.SaveStorageDefaults(ctx, tenkisandbox.StorageDefaults{ WorkspaceID: defaults.WorkspaceID, Mounts: mounts, }, defaults.Revision) ``` **CLI** ```bash tenki sandbox storage defaults set \ --mount datasets/imagenet:/mnt/imagenet:ro \ --mount media:/mnt/media tenki sandbox storage defaults get ``` `--mount` takes `[/]:[:ro|rw]`. `defaults set --clear` removes every default. Saving replaces the whole list, so include the defaults you want to keep, with their IDs. The CLI keeps the ID of a default whose path did not change. `tenki sandbox storage defaults get` shows each default's ID, which you need to change or skip a default when you create a sandbox. ### Mount at create Mounts you pass when you create a sandbox are added to the workspace defaults. You can also change a default's path, prefix, or access for this sandbox, leave one default out, or skip all of them. **TypeScript** ```typescript const session = await sandbox.createAndWait({ storageMounts: [ { connectionName: "media", prefix: "raw", mountPath: "/mnt/raw", access: "READ_ONLY" }, { defaultMountId: imagenetDefaultId, mountPath: "/workspace/imagenet" }, ], excludedDefaultMountIds: [otherDefaultId], }); ``` Pass `disableStorageDefaults: true` to skip every default. **Python** ```python from tenki import Sandbox, StorageMountSpec sb = Sandbox.create( storage_mounts=[ StorageMountSpec(connection_name="media", prefix="raw", mount_path="/mnt/raw", access="READ_ONLY"), StorageMountSpec(default_mount_id=imagenet_default_id, mount_path="/workspace/imagenet"), ], excluded_default_mount_ids=[other_default_id], ) ``` Pass `disable_storage_defaults=True` to skip every default. **Go** ```go prefix, readOnly := "raw", tenkisandbox.StorageAccessReadOnly session, err := client.Create( ctx, tenkisandbox.WithStorageMount(tenkisandbox.StorageMountSpec{ ConnectionName: "media", Prefix: &prefix, MountPath: "/mnt/raw", Access: &readOnly, }), tenkisandbox.WithStorageMount(tenkisandbox.StorageMountSpec{ DefaultMountID: imagenetDefaultID, MountPath: "/workspace/imagenet", }), tenkisandbox.WithoutDefaultStorageMount(otherDefaultID), ) ``` Pass `tenkisandbox.WithoutStorageDefaults()` to skip every default. **CLI** ```bash tenki sandbox create \ --storage-mount media/raw:/mnt/raw:ro \ --storage-default :/workspace/imagenet \ --exclude-storage-default ``` When you change a default, the fields you leave out keep the default's values. In the SDKs, an empty prefix mounts the whole bucket, and an empty access uses the connection's default access (`StorageAccessDefault` in Go). | CLI flag | Value | | --------------------------- | ------------------------------------------------------------------------- | | `--storage-mount` | `[/]:[:ro\|rw]`. Repeat for more than one. | | `--storage-default` | `[/[]][:][:ro\|rw\|default]`. Repeatable. | | `--exclude-storage-default` | ID of a default mount to leave out. Repeat for more than one. | | `--no-storage-defaults` | Skip every default mount. | In `--storage-default`, a bare `/` mounts the whole bucket, and `default` uses the connection's default access, as in `--storage-default /:default`. `--storage-mount` accepts a connection name or ID. In the dashboard, the **Storage mounts** section of the launch dialog for a published template lists the workspace defaults. Edit or remove a row, or add one, before you launch. If a mount fails while the sandbox starts, the sandbox still starts. That mount shows `failed` with the reason in `last_error`, such as `bucket_unreachable` or `credentials_unavailable`. Check with `tenki sandbox storage mounts list `, fix the cause, and [attach it again](#attach-and-detach) at the same path. A forked sandbox gets the source's mounts instead of the current defaults. Snapshots do not contain bucket data. A sandbox restored from a snapshot is a new sandbox, so it gets the workspace defaults and any mounts you pass. While Tenki takes a snapshot of a running sandbox, it unmounts the sandbox's storage mounts and mounts them again afterwards. The snapshot fails if a program has files open on a mount or an upload cannot finish, so close files on the mounts first. Pausing a sandbox also unmounts its storage mounts, and resuming it mounts them again. ### Attach and detach Attach and detach mounts on a running sandbox. **TypeScript** ```typescript const mount = await session.attachStorageMount({ connectionName: "media", prefix: "raw", mountPath: "/mnt/media" }); const mounts = await session.listStorageMounts(); await session.detachStorageMount(mount.id); ``` **Python** ```python mount = sb.attach_storage_mount(StorageMountSpec(connection_name="media", prefix="raw", mount_path="/mnt/media")) mounts = sb.list_storage_mounts() sb.detach_storage_mount(mount.id) ``` **Go** ```go prefix := "raw" mount, err := session.AttachStorageMount(ctx, tenkisandbox.StorageMountSpec{ ConnectionName: "media", Prefix: &prefix, MountPath: "/mnt/media", }) mounts, err := session.ListStorageMounts(ctx) err = session.DetachStorageMount(ctx, mount.ID) ``` **CLI** ```bash tenki sandbox storage mounts attach --connection media --path /mnt/media --prefix raw --access ro tenki sandbox storage mounts list tenki sandbox storage mounts detach ``` These commands take the session ID or name as their first argument. Without `--access`, the mount uses the connection's default access. In the dashboard, open the sandbox, click **Attach** under **Storage mounts**, fill in the connection, mount path, bucket prefix, and access, and click **Attach storage**. To detach, click the detach button next to the mount. Attach returns once the folder is mounted or the mount has failed. A failed attach returns an error with the reason, and the mount stays listed as `failed`. Attaching again at the same path replaces the failed mount. If someone detaches the mount before the attach finishes, attach raises `StorageMountAttachCancelledError` (Go: `ErrStorageMountAttachCancelled`) and the mount ends detached. If the mount fails before the attach finishes, attach returns that failure like any failed attach. While a detach runs, the mount shows `unmounting`, which the dashboard labels **Detaching**. Detach fails with `guest_unmount_failed` while a program in the sandbox has files open on the mount. It then waits up to a minute for pending uploads. If an upload cannot finish, detach fails with `writeback_failed`, and the folder stays mounted so no data is lost. After either failure the mount shows `failed`, but the folder is still mounted. Fix the cause and detach again. If Tenki cannot reach the sandbox's host, detach returns an error and the mount stays `unmounting`; detach again to retry. Terminating a sandbox also waits for pending uploads, up to a minute for each mount. Writes that still cannot be uploaded stay on the host for 7 days, then Tenki discards them. Contact Tenki support with the session ID and the mount ID within that time if you need them. Detach first when you need to know that every write reached the bucket. | State | Meaning | | --------------------- | ------------------------------------------------------------------------------------------------------------------ | | `pending`, `mounting` | Tenki is setting up the mount. | | `mounted` | The folder is ready. | | `unmounting` | A detach is running. If it could not reach the host, the mount stays here until you detach again. | | `failed` | Mounting or detaching failed, and `last_error` has the reason. After a failed detach the folder stays mounted. | | `detached` | Tenki detached the mount, and `last_error` is `detached_by_tenki`. Mounts you detached yourself drop off the list. | ## Limits A storage mount is not a local disk, and some file operations behave differently: * Renaming a file copies the object and deletes the old one, so renaming large files or folders is slow. * There are no file locks, hard links, or ownership changes. * Changes that other clients make to the bucket can take up to 5 minutes to appear, because the mount caches directory listings. * When two writers change the same object, the last upload wins. * Writes go to a cache on the host and upload about a second after the file is closed. Each mount's cache holds 2 GiB, so writing more than that before closing files fails with "No space left on device". * Each mount's host process gets at most one CPU core and 512 MiB of memory. Tenki checks these rules when you save a connection or a mount: * A workspace can have up to 100 storage connections and up to 16 default mounts. * A sandbox can have up to 16 storage mounts. * Mount paths are absolute, up to 255 characters, and cannot be `/` or sit under `/bin`, `/boot`, `/dev`, `/etc`, `/lib`, `/lib64`, `/proc`, `/root`, `/run`, `/sbin`, `/sys`, `/usr`, or `/var`. Two mounts in one sandbox cannot share a path or nest inside each other, and a storage mount cannot overlap a persistent volume's mount path. * Prefixes are up to 1024 characters, and leading and trailing slashes are ignored. Tenki rejects `*`, `?`, and `$`, because AWS policies read them as wildcards and variables. It also rejects backslashes and empty, `.`, or `..` path segments. * Connection names start with a letter and use letters, digits, spaces, `_`, `.`, and `-`, up to 128 characters. * Bucket names are 3 to 63 lowercase letters, digits, dots, and hyphens, and start and end with a letter or digit. * Endpoints are public `https://` URLs with no path, as described under [S3-compatible](#s3-compatible). * Endpoint and region changes are rejected while running sandboxes mount the connection, so detach those mounts first. * A mount that fails while a sandbox starts leaves the sandbox running, with that mount `failed`. ## Troubleshooting **A mount shows `failed`.** Run `tenki sandbox storage mounts list ` and read the error column, or `last_error` with `--json`: | `last_error` | What to do | | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | `bucket_unreachable` | Tenki could not reach the bucket. [Test the connection](#test-a-connection) for the exact reason. | | `credentials_unavailable` | Tenki could not get credentials for the mount. Test the connection, fix the cause, and attach again. | | `credentials_revoked` | A secret the connection reads was revoked while the mount ran. Update the connection and attach again. | | `credentials_expired`, `credential_refresh_failed` | Renewing the mount's credentials failed. Test the connection, then attach again. | | `writeback_failed` | An upload did not finish. Check that the key or role can still write to the bucket. | | `host_kept_cache_limit` | The host holds too much data that earlier mounts could not upload. Contact Tenki support. | | `connection_disabled`, `detached_by_tenki` | Tenki disabled the connection or detached the mount. See [Disabled by Tenki](#disabled-by-tenki). | For any other reason, contact Tenki support with the session ID, the mount ID, and the reason. **A storage command fails with `storage mounts are not enabled for this workspace`.** Storage mounts are in preview. Contact Tenki support to turn them on for the workspace. **Creating a keyless connection fails with `keyless AWS role connections are not available here; use access keys instead`.** This Tenki deployment has no OIDC issuer for AWS to trust. Use [AWS access keys](#aws-access-keys) instead. **Creating, updating, or deleting a connection fails with `API key requires Secrets access`.** Create an API key with **Allow Secrets access**, or log in to the CLI again and select it. Your account also needs workspace-edit permission. **Saving a connection fails with `secret ... already exists; select it instead of entering a value`.** A secret with the name Tenki would create already exists, and Tenki did not create it for this connection. Pick that secret with **Use secret** or `---secret`, or enter the value under a different connection name. **Create or attach fails with a storage connection not found error.** A mount names a connection that does not exist in the workspace. Connection names match without regard to case. The SDKs raise `StorageConnectionNotFoundError`, or return `ErrStorageConnectionNotFound` in Go; see the [SDK reference](https://tenki.cloud/docs/sandbox/sdk.md#errors). **Create or attach fails with `storage connection "" was disabled by Tenki; contact support`.** Tenki disabled the connection; see [Disabled by Tenki](#disabled-by-tenki). The SDKs raise `StorageConnectionDisabledError`, or return `ErrStorageConnectionDisabled` in Go. **Saving fails with `storage prefix must not contain *, ? or $`.** Remove the wildcard and mount the nearest parent prefix instead. **An update fails with `endpoint and region cannot change while 2 mounts use this connection; detach them first`.** Detach the listed mounts, update the connection, and attach them again. **Writes fail with "No space left on device".** The mount's 2 GiB upload cache is full of files that are still open or not uploaded yet. Close files sooner, or write large outputs in smaller parts.