Stores¶
Stores are implementations of the AsyncKeyValue protocol that provide actual
storage backends.
Memory Store¶
In-memory key-value store, useful for testing and development.
MemoryStore
¶
Bases: BaseDestroyStore, BaseDestroyCollectionStore, BaseEnumerateCollectionsStore, BaseEnumerateKeysStore
A fixed-size in-memory key-value store using TLRU (Time-aware Least Recently Used) cache.
__init__
¶
Initialize a fixed-size in-memory store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
max_entries_per_collection
|
int | None
|
The maximum number of entries per collection. Defaults to no limit. |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
seed
|
SEED_DATA_TYPE | None
|
Optional seed data to pre-populate the store. Format: {collection: {key: {field: value, ...}}}. Each value must be a mapping (dict) that will be stored as the entry's value. Seeding occurs lazily when each collection is first accessed. |
None
|
Disk Store¶
Persistent disk-based key-value store using DiskCache.
DiskStore
¶
Bases: BaseContextManagerStore, BaseStore
A disk-based store that uses the diskcache library to store data.
__init__
¶
__init__(
*,
disk_cache=None,
directory=None,
max_size=None,
default_collection=None,
auto_create=True,
)
Initialize the disk store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
disk_cache
|
Cache | None
|
An existing diskcache Cache instance to use. If provided, the store will not manage the cache's lifecycle (will not close it). The caller is responsible for managing the cache's lifecycle. |
None
|
directory
|
Path | str | None
|
The directory to use for the disk store. |
None
|
max_size
|
int | None
|
The maximum size of the disk store. |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
auto_create
|
bool
|
Whether to automatically create the directory if it doesn't exist. Defaults to True. When False, raises ValueError if the directory doesn't exist. |
True
|
FileTree Store¶
Directory-based store for visual inspection and testing.
FileTreeStore
¶
Bases: BaseStore
A file-tree based store using directories for collections and files for keys.
This store uses the native filesystem: - Each collection is a subdirectory under the base directory - Each key is stored as a JSON file named "{key}.json" - File contents contain the ManagedEntry serialized to JSON
Directory structure
{base_directory}/ {collection_1}/ {key_1}.json {key_2}.json {collection_2}/ {key_3}.json
By default, collections and keys are not sanitized. This means that filesystem limitations on path lengths and special characters may cause errors when trying to get and put entries.
To avoid issues, you may want to consider leveraging the FileTreeV1CollectionSanitizationStrategy
and FileTreeV1KeySanitizationStrategy strategies.
Security
This store includes the following security measures:
- Path validation: All file paths are validated to stay within the data directory.
The validation uses resolve() which follows symlinks, so symlink-based escapes
are also detected and blocked.
- Atomic writes: Files are written atomically using write-to-temp-then-rename with
fsync for durability. This ensures data is not corrupted on system crash.
Limitations
- No file locking: Concurrent writes to the same key from multiple processes may cause data loss (last write wins). Single-writer or external locking is recommended for multi-process scenarios.
- No built-in cleanup of expired entries. Expired entries are only filtered out when read via get() or similar methods.
- Performance may degrade with very large numbers of keys per collection due to filesystem directory entry limits.
__init__
¶
__init__(
*,
data_directory,
metadata_directory=None,
default_collection=None,
serialization_adapter=None,
key_sanitization_strategy=None,
collection_sanitization_strategy=None,
auto_create=True,
)
Initialize the file-tree store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_directory
|
Path | str
|
The base directory to use for storing collections and keys. |
required |
metadata_directory
|
Path | str | None
|
The directory to use for storing metadata. Defaults to data_directory. |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
serialization_adapter
|
SerializationAdapter | None
|
The serialization adapter to use for the store. |
None
|
key_sanitization_strategy
|
SanitizationStrategy | None
|
The sanitization strategy to use for keys. |
None
|
collection_sanitization_strategy
|
SanitizationStrategy | None
|
The sanitization strategy to use for collections. |
None
|
auto_create
|
bool
|
Whether to automatically create directories if they don't exist. Defaults to True. When False, raises ValueError if a directory doesn't exist. |
True
|
Redis Store¶
Redis-backed key-value store.
RedisStore
¶
Bases: BaseDestroyStore, BaseEnumerateKeysStore, BaseContextManagerStore, BaseStore
Redis-based key-value store.
__init__
¶
__init__(
*,
client=None,
default_collection=None,
url=None,
host="localhost",
port=6379,
db=0,
password=None,
)
Initialize the Redis store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
Redis | None
|
An existing Redis client to use. If provided, the store will not manage the client's lifecycle (will not close it). The caller is responsible for managing the client's lifecycle. |
None
|
url
|
str | None
|
Redis URL (e.g., redis://localhost:6379/0). |
None
|
host
|
str
|
Redis host. Defaults to localhost. |
'localhost'
|
port
|
int
|
Redis port. Defaults to 6379. |
6379
|
db
|
int
|
Redis database number. Defaults to 0. |
0
|
password
|
str | None
|
Redis password. Defaults to None. |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
DynamoDB Store¶
AWS DynamoDB-backed key-value store.
DynamoDBStore
¶
Bases: BaseContextManagerStore, BaseStore
DynamoDB-based key-value store.
This store uses a single DynamoDB table with a composite primary key: - collection (partition key) - key (sort key)
__init__
¶
__init__(
*,
client: AioBaseClient,
table_name: str,
default_collection: str | None = None,
table_config: dict[str, Any] | None = None,
auto_create: bool = True,
) -> None
__init__(
*,
table_name: str,
region_name: str | None = None,
endpoint_url: str | None = None,
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_session_token: str | None = None,
default_collection: str | None = None,
table_config: dict[str, Any] | None = None,
auto_create: bool = True,
) -> None
__init__(
*,
client=None,
table_name,
region_name=None,
endpoint_url=None,
aws_access_key_id=None,
aws_secret_access_key=None,
aws_session_token=None,
default_collection=None,
table_config=None,
auto_create=True,
)
Initialize the DynamoDB store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
AioBaseClient | None
|
The DynamoDB client to use. If provided, the store will not manage the client's lifecycle (will not enter/exit its context manager). The caller is responsible for managing the client's lifecycle and must ensure the client is already entered. |
None
|
table_name
|
str
|
The name of the DynamoDB table to use. |
required |
region_name
|
str | None
|
AWS region name. Defaults to None (uses AWS default). |
None
|
endpoint_url
|
str | None
|
Custom endpoint URL (useful for local DynamoDB). Defaults to None. |
None
|
aws_access_key_id
|
str | None
|
AWS access key ID. Defaults to None (uses AWS default credentials). |
None
|
aws_secret_access_key
|
str | None
|
AWS secret access key. Defaults to None (uses AWS default credentials). |
None
|
aws_session_token
|
str | None
|
AWS session token. Defaults to None (uses AWS default credentials). |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
table_config
|
dict[str, Any] | None
|
Additional configuration to pass to create_table(). Merged with defaults. Examples: SSESpecification, Tags, StreamSpecification, etc. Note: Critical parameters (TableName, KeySchema, AttributeDefinitions, BillingMode) cannot be overridden as they are required for store operation. |
None
|
auto_create
|
bool
|
Whether to automatically create the table if it doesn't exist. Defaults to True. When False, raises ValueError if the table doesn't exist. |
True
|
S3 Store¶
AWS S3-backed key-value store.
S3Store
¶
Bases: BaseContextManagerStore, BaseStore
AWS S3-based key-value store.
This store uses AWS S3 to store key-value pairs as objects. Each entry is stored as a separate S3 object with the path format: {collection}/{key}. The ManagedEntry is serialized to JSON and stored as the object body. TTL information is stored in S3 object metadata and checked client-side during retrieval (S3 lifecycle policies can be configured separately for background cleanup, but don't provide atomic TTL+retrieval).
By default, collections and keys are not sanitized. This means you must ensure that the combined "{collection}/{key}" path does not exceed S3's 1024-byte limit when UTF-8 encoded.
To handle long collection or key names, use the S3CollectionSanitizationStrategy and S3KeySanitizationStrategy which will hash values exceeding the byte limit.
Example
Basic usage with automatic AWS credentials:
async with S3Store(bucket_name="my-kv-store") as store: ... await store.put(key="user:123", value={"name": "Alice"}, ttl=3600) ... user = await store.get(key="user:123")
With sanitization for long keys/collections:
async with S3Store( ... bucket_name="my-kv-store", ... collection_sanitization_strategy=S3CollectionSanitizationStrategy(), ... key_sanitization_strategy=S3KeySanitizationStrategy(), ... ) as store: ... await store.put(key="very_long_key" * 100, value={"data": "test"})
With custom AWS credentials:
async with S3Store( ... bucket_name="my-kv-store", ... region_name="us-west-2", ... aws_access_key_id="...", ... aws_secret_access_key="...", ... ) as store: ... await store.put(key="config", value={"setting": "value"})
For local testing with LocalStack:
async with S3Store( ... bucket_name="test-bucket", ... endpoint_url="http://localhost:4566", ... ) as store: ... await store.put(key="test", value={"data": "test"})
__init__
¶
__init__(
*,
client: AioBaseClient,
bucket_name: str,
default_collection: str | None = None,
collection_sanitization_strategy: SanitizationStrategy | None = None,
key_sanitization_strategy: SanitizationStrategy | None = None,
) -> None
__init__(
*,
bucket_name: str,
region_name: str | None = None,
endpoint_url: str | None = None,
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_session_token: str | None = None,
default_collection: str | None = None,
collection_sanitization_strategy: SanitizationStrategy | None = None,
key_sanitization_strategy: SanitizationStrategy | None = None,
) -> None
__init__(
*,
client=None,
bucket_name,
region_name=None,
endpoint_url=None,
aws_access_key_id=None,
aws_secret_access_key=None,
aws_session_token=None,
default_collection=None,
collection_sanitization_strategy=None,
key_sanitization_strategy=None,
)
Initialize the S3 store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
AioBaseClient | None
|
The S3 client to use. Defaults to None (creates a new client). |
None
|
bucket_name
|
str
|
The name of the S3 bucket to use. |
required |
region_name
|
str | None
|
AWS region name. Defaults to None (uses AWS default). |
None
|
endpoint_url
|
str | None
|
Custom endpoint URL (useful for LocalStack/MinIO). Defaults to None. |
None
|
aws_access_key_id
|
str | None
|
AWS access key ID. Defaults to None (uses AWS default credentials). |
None
|
aws_secret_access_key
|
str | None
|
AWS secret access key. Defaults to None (uses AWS default credentials). |
None
|
aws_session_token
|
str | None
|
AWS session token. Defaults to None (uses AWS default credentials). |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
collection_sanitization_strategy
|
SanitizationStrategy | None
|
Strategy for sanitizing collection names. Defaults to None (no sanitization). |
None
|
key_sanitization_strategy
|
SanitizationStrategy | None
|
Strategy for sanitizing keys. Defaults to None (no sanitization). |
None
|
Elasticsearch Store¶
Elasticsearch-backed key-value store.
ElasticsearchStore
¶
Bases: BaseEnumerateCollectionsStore, BaseEnumerateKeysStore, BaseDestroyCollectionStore, BaseCullStore, BaseContextManagerStore, BaseStore
An Elasticsearch-based store.
Stores collections in their own indices and stores values in Flattened fields.
This store has specific restrictions on what is allowed in keys and collections. Keys and collections are not sanitized by default which may result in errors when using the store.
To avoid issues, you may want to consider leveraging the ElasticsearchV1KeySanitizationStrategy and
ElasticsearchV1CollectionSanitizationStrategy strategies.
__init__
¶
__init__(
*,
elasticsearch_client=None,
url=None,
api_key=None,
index_prefix,
default_collection=None,
key_sanitization_strategy=None,
collection_sanitization_strategy=None,
auto_create=True,
)
Initialize the elasticsearch store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
elasticsearch_client
|
AsyncElasticsearch | None
|
The elasticsearch client to use. If provided, the store will not manage the client's lifecycle (will not close it). The caller is responsible for managing the client's lifecycle. |
None
|
url
|
str | None
|
The url of the elasticsearch cluster. |
None
|
api_key
|
str | None
|
The api key to use. |
None
|
index_prefix
|
str
|
The index prefix to use. Collections will be prefixed with this prefix. |
required |
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
key_sanitization_strategy
|
SanitizationStrategy | None
|
The sanitization strategy to use for keys. |
None
|
collection_sanitization_strategy
|
SanitizationStrategy | None
|
The sanitization strategy to use for collections. |
None
|
auto_create
|
bool
|
Whether to automatically create indices if they don't exist. Defaults to True. When False, raises ValueError if an index doesn't exist. |
True
|
MongoDB Store¶
MongoDB-backed key-value store.
MongoDBStore
¶
Bases: BaseDestroyCollectionStore, BaseContextManagerStore, BaseStore
MongoDB-based key-value store using pymongo.
Stores collections as MongoDB collections and stores values in document fields.
By default, collections are not sanitized. This means that there are character and length restrictions on collection names that may cause errors when trying to get and put entries.
To avoid issues, you may want to consider leveraging the MongoDBV1CollectionSanitizationStrategy strategy.
__init__
¶
__init__(
*,
client=None,
url=None,
db_name=None,
coll_name=None,
default_collection=None,
collection_sanitization_strategy=None,
auto_create=True,
)
Initialize the MongoDB store.
Values are stored as native BSON dictionaries for better query support and performance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
AsyncMongoClient[dict[str, Any]] | None
|
The MongoDB client to use (mutually exclusive with url). If provided, the store will not manage the client's lifecycle (will not enter/exit its context manager or close it). The caller is responsible for managing the client's lifecycle. |
None
|
url
|
str | None
|
The url of the MongoDB cluster (mutually exclusive with client). |
None
|
db_name
|
str | None
|
The name of the MongoDB database. |
None
|
coll_name
|
str | None
|
The name of the MongoDB collection. |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
collection_sanitization_strategy
|
SanitizationStrategy | None
|
The sanitization strategy to use for collections. |
None
|
auto_create
|
bool
|
Whether to automatically create collections if they don't exist. Defaults to True. When False, raises ValueError if a collection doesn't exist. |
True
|
Valkey Store¶
Valkey-backed key-value store (Redis-compatible).
ValkeyStore
¶
Bases: BaseContextManagerStore, BaseStore
Valkey-based key-value store (Redis protocol compatible).
Supports both standalone (GlideClient) and cluster (GlideClusterClient) deployments.
__init__
¶
__init__(
*,
client=None,
default_collection=None,
host="localhost",
port=6379,
db=0,
username=None,
password=None,
)
Initialize the Valkey store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
BaseClient | None
|
An existing Valkey client to use (GlideClient or GlideClusterClient). If provided, the store will not manage the client's lifecycle (will not close it). The caller is responsible for managing the client's lifecycle. |
None
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
host
|
str
|
Valkey host. Defaults to localhost. |
'localhost'
|
port
|
int
|
Valkey port. Defaults to 6379. |
6379
|
db
|
int
|
Valkey database number. Defaults to 0. |
0
|
username
|
str | None
|
Valkey username. Defaults to None. |
None
|
password
|
str | None
|
Valkey password. Defaults to None. |
None
|
Note
When using a cluster client, the host/port/db parameters are ignored. You must provide a pre-configured GlideClusterClient instance.
Memcached Store¶
Memcached-backed key-value store.
MemcachedStore
¶
Bases: BaseDestroyStore, BaseContextManagerStore, BaseStore
Memcached-based key-value store using aiomcache.
By default, keys are not sanitized. This means that there are character and length restrictions on keys that may cause errors when trying to get and put entries.
To avoid issues, you may want to consider leveraging the MemcachedV1KeySanitizationStrategy strategy.
__init__
¶
__init__(
*,
client=None,
host="127.0.0.1",
port=11211,
default_collection=None,
key_sanitization_strategy=None,
)
Initialize the Memcached store.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
client
|
Client | None
|
An existing aiomcache client to use. If provided, the store will not manage the client's lifecycle (will not close it). The caller is responsible for managing the client's lifecycle. |
None
|
host
|
str
|
Memcached host. Defaults to 127.0.0.1. |
'127.0.0.1'
|
port
|
int
|
Memcached port. Defaults to 11211. |
11211
|
default_collection
|
str | None
|
The default collection to use if no collection is provided. |
None
|
key_sanitization_strategy
|
SanitizationStrategy | None
|
The sanitization strategy to use for keys. |
None
|
Null Store¶
A no-op store that doesn't persist anything, useful for testing.
NullStore
¶
Bases: BaseStore
Null object pattern store that accepts all operations but stores nothing.