DigitalOcean Spaces Setup

Hardhat Flow uses DigitalOcean Spaces (S3-compatible) for all file storage: photos, thumbnails, avatars, bid PDFs, and per-tenant database backups. Local development uses the filesystem by default; set USE_S3=True to activate Spaces.

---

1. Create a Spaces Bucket

1. Log in to the DigitalOcean control panel.
2. Navigate to Spaces Object Storage → Create a Space.
3. Choose a region (e.g. nyc3). Use the same region for all environments to avoid cross-region egress fees.
4. Set access to Restricted (private). The application manages presigned URLs — no public ACL needed.
5. Note the bucket name and endpoint URL (e.g. https://nyc3.digitaloceanspaces.com).

---

2. Create Spaces Access Keys

1. Control panel → API → Spaces Keys → Generate New Key.
2. Give the key a descriptive name (e.g. hardhat-flow-production).
3. Copy the Key (Access Key ID) and Secret immediately — the secret is not shown again.

---

3. Configure CORS (if the frontend uploads directly)

If the frontend needs direct-upload access to Spaces, add a CORS rule on the bucket:

[
  {
    "AllowedHeaders": ["*"],
    "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
    "AllowedOrigins": ["https://your-app-domain.com"],
    "MaxAgeSeconds": 3000
  }
]

The current implementation uploads via the Django API (server-side), so CORS is not required unless direct-upload is added later.

---

4. Environment Variables

Set these in the DigitalOcean App Platform environment (or your .env for local Spaces testing):

Variable	Example	Notes
USE_S3	True	Activates the Spaces backend
AWS_ACCESS_KEY_ID	DO00XXXX...	Spaces access key
AWS_SECRET_ACCESS_KEY	...	Spaces secret key
AWS_STORAGE_BUCKET_NAME	hardhat-flow-prod	Bucket name
AWS_S3_ENDPOINT_URL	https://nyc3.digitaloceanspaces.com	Regional endpoint
AWS_S3_REGION_NAME	nyc3	Must match the bucket region
AWS_S3_CUSTOM_DOMAIN	hardhat-flow-prod.nyc3.cdn.digitaloceanspaces.com	Optional CDN domain
AWS_QUERYSTRING_EXPIRE	3600	Presigned URL TTL in seconds (default: 3600)

On DigitalOcean App Platform, do not declare AWS_S3_ENDPOINT_URL again under an individual component (web, worker, beat) as an encrypted secret unless you actually set a value there. An empty component-level entry overrides app-wide variables with a blank string, and Django then fails when saving to Spaces with ValueError: Invalid endpoint from boto3. The project .do/app.yaml sets the regional endpoint once at app scope (plain value, not a secret).

---

5. File Organization

All files are prefixed by tenant slug to enforce isolation:

Type	Storage path
Photos	{tenant_slug}/{entity_type}/{entity_id}/{uuid}.{ext}
Thumbnails	{tenant_slug}/{entity_type}/{entity_id}/thumbnails/{uuid}.jpg
Bid PDFs	{tenant_slug}/bids/{bid_id}/bid-{YYYYMMDD}.pdf
Avatars	avatars/{user_id}/{uuid}.{ext}
DB Backups	backups/{tenant_slug}/{timestamp}.sql.gz

All files are stored with private ACL. URLs returned by the API are presigned and expire after AWS_QUERYSTRING_EXPIRE seconds. The storage path (not the presigned URL) is what is persisted in the database; a fresh presigned URL is generated on each API read.

---

6. Verify the Setup

After setting env vars and deploying:

# Upload a test photo via the API and check the response includes a presigned URL
curl -X POST https://your-app/{slug}/api/v1/photos/bids/{bid_id}/ \
  -H "Authorization: Bearer <token>" \
  -F "[email protected]"

# The file_url in the response should point to your Spaces bucket with a ?X-Amz-Signature= query param

Check the DigitalOcean Spaces browser to confirm the file appears at the expected prefix.

---

7. Object Versioning & Restore

Object versioning is enabled on hardhat-flow-prod so overwritten or deleted files can be recovered. A lifecycle rule deletes noncurrent versions after 90 days to prevent unbounded storage growth.

Enable versioning on a bucket

Run once after provisioning a new bucket (requires USE_S3=True in environment):

python manage.py enable_spaces_versioning

The command is idempotent — safe to re-run. If versioning and the lifecycle rule are already correct, it exits with no API mutations. Options:

Flag	Default	Description
--bucket <name>	AWS_STORAGE_BUCKET_NAME	Target a specific bucket
--noncurrent-days <n>	90	Days before old versions are deleted
--skip-lifecycle	—	Enable versioning only; skip lifecycle rule
--dry-run	—	Print intended actions without API calls
--force	—	Run even when USE_S3=False

Verify:

aws --endpoint-url https://nyc3.digitaloceanspaces.com \
    s3api get-bucket-versioning --bucket hardhat-flow-prod
# Expected: {"Status": "Enabled"}

aws --endpoint-url https://nyc3.digitaloceanspaces.com \
    s3api get-bucket-lifecycle-configuration --bucket hardhat-flow-prod
# Expected: rule "expire-noncurrent-versions" with NoncurrentDays: 90

Restore an overwritten or deleted file

1. List versions (use the storage key from the database, e.g. acme/jobs/42/abc.jpg):

aws --endpoint-url https://nyc3.digitaloceanspaces.com \
    s3api list-object-versions \
    --bucket hardhat-flow-prod \
    --prefix acme/jobs/42/abc.jpg

1. Download a prior version — copy the VersionId from the Versions list:

aws --endpoint-url https://nyc3.digitaloceanspaces.com \
    s3api get-object \
    --bucket hardhat-flow-prod \
    --key acme/jobs/42/abc.jpg \
    --version-id <version-id> \
    restored.jpg

1. Re-upload to make it the current version:

aws --endpoint-url https://nyc3.digitaloceanspaces.com \
    s3 cp restored.jpg s3://hardhat-flow-prod/acme/jobs/42/abc.jpg

To restore a deleted file, find the DeleteMarker entry in list-object-versions output and delete that marker:

aws --endpoint-url https://nyc3.digitaloceanspaces.com \
    s3api delete-object \
    --bucket hardhat-flow-prod \
    --key acme/jobs/42/abc.jpg \
    --version-id <delete-marker-version-id>

This makes the previous version current again without re-uploading.