Hosting

AWS Bedrock

Extraction and embeddings in your own AWS account.

llms.txt

Run extraction on Claude and embeddings on Cohere in your own AWS account, signed with AWS credentials instead of an API key. On ECS the server uses the task role, so no keys live anywhere.

Step 1: Enable model access

In the AWS console open Bedrock → Model access and enable the models you will use, in your region: Claude Haiku 5.5 for extraction and Cohere Embed English v3 for embeddings by default. This is the step people miss: without it every call is denied. The role or user the server runs as needs bedrock:InvokeModel on both.

Step 2: Run with the bedrock providers

The Docker image already has the bedrock feature. From source, build with it:

Terminal
cargo install mentedb-server --features bedrock
export AWS_REGION=us-east-1
export MENTEDB_LLM_PROVIDER=bedrock            # extraction on Claude
export MENTEDB_EMBEDDING_PROVIDER=bedrock      # embeddings on Cohere
mentedb-server --data-dir ./data --auto-extract
SettingDefault
MENTEDB_LLM_MODELus.anthropic.claude-haiku-5-5
MENTEDB_LLM_REGIONAWS_REGION, else us-east-1
MENTEDB_EMBEDDING_MODELcohere.embed-english-v3
MENTEDB_EMBEDDING_REGIONAWS_REGION, else us-east-1

MENTEDB_EMBEDDING_MODEL accepts cohere.embed-english-v3, cohere.embed-multilingual-v3 (both 1024 dims, sent in batches of 96), amazon.titan-embed-text-v2:0 (1024) and amazon.titan-embed-text-v1 (1536). Any other value stops the server at startup. At startup the server embeds one test string and logs a warning if Bedrock refuses, so a missing permission shows up in the log before the first write.

Credentials

Bedrock calls are signed with the first credentials found, in order:

  1. Static keys: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, plus AWS_SESSION_TOKEN for temporary credentials.
  2. The container credentials endpoint used by ECS task roles and EKS Pod Identity (AWS_CONTAINER_CREDENTIALS_RELATIVE_URI or AWS_CONTAINER_CREDENTIALS_FULL_URI, which the platform sets). These credentials are temporary; the server caches them and fetches new ones shortly before they expire.

AWS profiles, SSO sessions and EC2 instance metadata are not read. On EC2 or a laptop, export keys (for example with aws configure export-credentials --format env), or run in a container with a role.

On ECS with a task role

Give the task role bedrock:InvokeModel on the two models, and set only environment variables in the task definition:

JSON
"environment": [
  { "name": "AWS_REGION", "value": "us-east-1" },
  { "name": "MENTEDB_LLM_PROVIDER", "value": "bedrock" },
  { "name": "MENTEDB_EMBEDDING_PROVIDER", "value": "bedrock" },
  { "name": "MENTEDB_AUTO_EXTRACT", "value": "1" }
]

With Bedrock embeddings the image never downloads the local model. Point the container health check at /v1/ready, which also checks that the embedder answers.

Notes

  • MENTEDB_LLM_MODEL takes a Bedrock model id or inference profile (like us.anthropic.claude-sonnet-5-5), not an Anthropic API model name.
  • A data directory keeps the embedder it was filled with. Moving from candle (384 dims) to Bedrock (1024) means starting a fresh directory or re-embedding.