Feature: Add multi-config

[vietdevne]

Add Multi-Configuration Support for Cross-Account DynamoDB Access

🎯 Overview

This PR introduces multi-configuration support to Dynamoid, enabling applications to connect to multiple DynamoDB instances across different AWS accounts, regions, and environments within a single application. This feature is particularly valuable for:

• Cross-account data access patterns
• Multi-region deployments
• Environment isolation (production/staging/analytics)
• Partner integrations requiring separate AWS credentials

🚀 Key Features

1. Multi-Configuration Registry

• Central registry to manage multiple DynamoDB configurations
• DSL-style configuration with block syntax
• Lazy-loaded adapter creation for optimal performance
• Thread-safe configuration management

2. Model-Level Configuration

• New dynamoid_config method to specify which configuration a model should use
• Automatic adapter selection based on model configuration
• Seamless integration with existing Dynamoid features
• Full backward compatibility with existing single-config setups

3. Isolated Connection Management

• Separate adapter instances for each configuration
• Independent connection pools and settings
• Configuration-specific namespaces for table isolation
• Proper error handling for unknown configurations

📝 Usage Examples

Basic Multi-Configuration Setup

# config/initializers/dynamoid.rb
Dynamoid.multi_configure do |config|
  # Primary application data
  config.add_config(:primary) do |c|
    c.access_key = ENV['PRIMARY_AWS_ACCESS_KEY']
    c.secret_key = ENV['PRIMARY_AWS_SECRET_KEY']
    c.region = 'us-east-1'
    c.namespace = 'myapp_primary'
  end

  # Analytics data in separate account
  config.add_config(:analytics) do |c|
    c.access_key = ENV['ANALYTICS_AWS_ACCESS_KEY']
    c.secret_key = ENV['ANALYTICS_AWS_SECRET_KEY']
    c.region = 'us-west-2'
    c.namespace = 'myapp_analytics'
  end

  # Cross-account partner data
  config.add_config(:partner) do |c|
    c.credentials = Aws::AssumeRoleCredentials.new(
      role_arn: ENV['PARTNER_ROLE_ARN'],
      role_session_name: 'dynamoid-partner'
    )
    c.region = 'eu-west-1'
    c.namespace = 'partner_shared'
  end
end

Model Configuration

# Uses primary configuration
class User
  include Dynamoid::Document
  
  dynamoid_config :primary
  
  field :name, :string
  field :email, :string
end

# Uses analytics configuration  
class PageView
  include Dynamoid::Document
  
  dynamoid_config :analytics
  
  field :url, :string
  field :user_id, :string
  field :timestamp, :datetime
end

# Uses partner configuration
class SharedData
  include Dynamoid::Document
  
  dynamoid_config :partner
  
  field :partner_id, :string
  field :content, :serialized
end

# Uses default configuration (backward compatibility)
class SystemLog
  include Dynamoid::Document
  
  # No dynamoid_config specified - uses default
  field :message, :string
end

🔧 Implementation Details

New Files Added:

• lib/dynamoid/multi_config.rb - Core multi-configuration functionality
• lib/dynamoid/errors.rb - Added UnknownConfiguration error

Files Modified:

• lib/dynamoid.rb - Added multi-config require and helper methods
• lib/dynamoid/document.rb - Added model-level configuration support
• lib/dynamoid/persistence.rb - Updated to use model-specific adapters
• lib/dynamoid/finders.rb - Updated to use model-specific adapters
• lib/dynamoid/criteria/chain.rb - Updated to use model-specific adapters
• lib/dynamoid/persistence/*.rb - Updated all persistence operations
• README.md - Comprehensive multi-config documentation

Architecture:

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   Model A       │───▶│  MultiConfig     │───▶│   Adapter A     │
│ (config: :prod) │    │   Registry       │    │ (prod config)   │
└─────────────────┘    │                  │    └─────────────────┘
                       │                  │    
┌─────────────────┐    │                  │    ┌─────────────────┐
│   Model B       │───▶│  - Configuration │───▶│   Adapter B     │
│ (config: :test) │    │    Storage       │    │ (test config)   │
└─────────────────┘    │  - Adapter       │    └─────────────────┘
                       │    Management    │    
┌─────────────────┐    │  - Error         │    ┌─────────────────┐
│   Model C       │───▶│    Handling      │───▶│ Default Adapter │
│ (no config)     │    │                  │    │ (main config)   │
└─────────────────┘    └──────────────────┘    └─────────────────┘

🧪 Test Coverage

• Unit Tests: 12 tests covering MultiConfig core functionality
• Integration Tests: 6 tests for Document integration
• E2E Tests: 3 tests for end-to-end scenarios
• Total: 21 tests, all passing ✅

🔄 Backward Compatibility

• 100% backward compatible - existing code continues to work unchanged
• Models without dynamoid_config automatically use the default configuration
• All existing Dynamoid features work seamlessly with multi-config
• No breaking changes to public APIs

📊 Benefits

1. Multi-tenant architectures - Separate data by customer/account
2. Compliance requirements - Data residency and isolation
3. Performance optimization - Region-specific data placement
4. Cost optimization - Different billing accounts for different workloads
5. Development workflows - Separate dev/staging/prod environments

🛡️ Error Handling

• Dynamoid::Errors::UnknownConfiguration for invalid configuration names
• Validation of configuration existence before adapter creation
• Graceful fallback to default configuration when appropriate
• Clear error messages with configuration troubleshooting information

📚 Documentation

• Comprehensive README section with examples and best practices
• Inline code documentation for all new methods
• Usage patterns for common scenarios (cross-account, multi-region, etc.)
• Migration guide for existing applications

🔍 Configuration Management

# List all configurations
Dynamoid::MultiConfig.configuration_names
# => [:primary, :analytics, :partner]

# Check if configuration exists
Dynamoid::MultiConfig.configuration_exists?(:primary)
# => true

# Get specific configuration
config = Dynamoid::MultiConfig.get_config(:primary)
config.region # => "us-east-1"

# Runtime configuration management
Dynamoid::MultiConfig.remove_config(:old_config)
Dynamoid::MultiConfig.clear_all

🎯 Use Cases Addressed

1. Enterprise applications connecting to multiple AWS accounts
2. SaaS platforms with customer-specific data isolation
3. Analytics pipelines with separate data warehouses
4. Partner integrations requiring cross-account access
5. Multi-region deployments with region-specific data

This implementation provides a robust, scalable solution for complex DynamoDB connectivity requirements while maintaining the simplicity and elegance that makes Dynamoid popular.

---

Breaking Changes: None ✅
Migration Required: No ✅
Backward Compatible: Yes ✅

[andrykonchin]

Related issues: #403, #831, #509

[thienhv-dev]

LGTM!

[andrykonchin]

First of all thank you for your interest to Dynamoid and I appreciate efforts you put into this big pull request.

As I mentioned in a comment above there are already a few discussions regarding functionality and interface this feature is supposed to have. I wish you have joined one of them before starting working on the code.

So have been thinking about this feature for quite a long time and have some opinion that would like to share with you.

The feature users were asking for is very close to the Rails's Multiple Databases (see the guide). As far as Dynamoid is trying to mimic Rails ActiveRecord's API as much as possible (to be sometimes a drop-in replacement) it's beneficial to review the Rails' multi database approach and borrow from the Rails' interface parts that work well for Dynamoid.

Multi config concept

How it's done in Rails and it completely makes sense for Dynamoid as well is to have not a complete fully fledged config per database, but only connection-related options like user/password/port/etc. In Dynamoid such options are access_key, secret_key, credentials etc (they are used at adapter instantiation and listed here). Config options specific to an application should be still configured globally for all DynamoDB regions and accounts, e.g. warn_on_scan, timestamps, store_attribute_with_nil_value, application_timezone etc. Probably sometimes it might be useful to have options like store_datetime_as_string configured separately for different DynamoDB instances when you work with legacy tables but such options usually can be configured per model field so it shouldn't be a problem.

This way we talk about not config but connection config/profile/etc. So Rails' terminology completely makes sense in Dynamoid too, e.g. method names like #connects_to, #connected_to.

For instance a model configuration may look like

class User
  include Dynamoid::Document

  connects_to :primary
  # ...
end

And configuration like this:

config do |c|
  c.access_key = ENV['PRIMARY_AWS_ACCESS_KEY']
  c.secret_key = ENV['PRIMARY_AWS_SECRET_KEY']
  c.region = 'us-east-1'
  c.namespace = 'myapp_primary'
end

config :analytics do |c|
   c.access_key = ENV['ANALYTICS_AWS_ACCESS_KEY']
   c.secret_key = ENV['ANALYTICS_AWS_SECRET_KEY']
   c.region = 'us-west-2'
   c.namespace = 'myapp_analytics'
end

Configuration Management

I would keep it as small as possible and add methods only when users ask for them. So instead of configuration_names, configuration_exists?, get_config, remove_config, clear_all I would simply keep methods similar to what Rails exposes (see documentation) and of cause names might be adopted to Dynamoid a bit:

• #connection_specification_name that returns :primary etc for a model class (e.g. User.connection_specification_name)
• #connection_db_config that returns just a config object for a model class (e.g. User.connection_db_config)

And I wouldn't expose Dynamoid::MultiConfig in public API. It makes renaming the class a breaking change what we would like avoid.

I've reviewed the pull request briefly and there might be more low-level remarks regarding particular line of code.

So once again thank you for your efforts and look forward for your next iteration on this pull request.

[taipa-2180]

lgtm

[vietdevne]

@andrykonchin
#918 (comment)

Thank you sincerely for your insightful and valuable comments. I will respectfully revise this pull request at my earliest convenience.

[ckhsponge]

@andrykonchin @vietdevne
I think I may have mentioned this before but my preference would be to allow full ARNs as table names in Dynamoid and then handle all the permissions using AWS policies. Policy permissions can span regions and accounts.

I do see the sometimes benefit of using access keys while developing, however. In production, keys should be avoided. When you create a key in AWS you'll see a long list of alternative authentication methods and reasons to use them.