Skip to content

Conversation

sav-norem
Copy link
Contributor

@sav-norem sav-norem commented Jul 5, 2024

Summary

Implements field projection capabilities for Redis OM queries, allowing users to retrieve only specific fields from models instead of
loading complete objects. This feature improves performance and reduces memory usage when working with large models or when only subset
of fields are needed.

New Features

.values() Method - Dictionary Results

Returns query results as dictionaries containing only the specified fields:

# Get specific fields as dictionaries
customers = Customer.find().values("first_name", "email")
# Returns: [{"first_name": "John", "email": "[email protected]"}]

# Get all fields as dictionaries
customers = Customer.find().values()
# Returns full objects as dictionaries

.only() Method - Partial Model Instances

Returns partial model instances that contain only the specified fields:

  # Get partial model instances
  customers = Customer.find().only("first_name", "email")
  for customer in customers:
      print(customer.first_name)  # ✓ Works - field was loaded
      print(customer.age)         # ✗ Raises AttributeError - field not loaded

Advanced Features

Nested Field Access

Both methods support accessing nested fields using double underscore notation:

  # Access nested JSON fields
  customers = Customer.find().values("name", "address__city", "address__zipcode")
  # Returns: [{"name": "John", "address__city": "NYC", "address__zipcode": "10001"}]

Type Conversion

  • .values() performs automatic type conversion (strings to integers, dates, etc.)
  • .only() maintains proper model field types and validation

Performance Optimizations

  • HashModel: Uses HMGET for efficient retrieval of specific hash fields
  • JsonModel: Uses JSON.GET with JSONPath expressions for efficient nested field extraction

Implementation Details

Core Changes

  • aredis_om/model/model.py: Added query builder methods and field projection logic
  • docs/models.md: Documentation with examples

Query Builder Integration

Both methods integrate seamlessly with existing query operations:

Customer.find(Customer.age > 25).sort_by("name").values("name", "email").all()
Customer.find().only("first_name").first()

Error Handling

  • Validates field names at query time
  • Provides clear error messages for invalid or non-existent fields
  • .only() raises AttributeError when accessing non-loaded fields

Breaking Changes

None - this is an additive feature that maintains full backward compatibility.

Documentation

  • Complete usage examples in docs/models.md
  • Covers basic usage, advanced features, and performance considerations
  • Includes nested field access patterns and best practices

Fixes #568

@sav-norem sav-norem marked this pull request as ready for review August 14, 2024 14:39
@sav-norem sav-norem requested review from slorello89 and banker August 14, 2024 14:39
@DennyD17
Copy link

Please add also exclude keyword, it'll be convenient

@DennyD17
Copy link

Are u going to finish it guys?

@DennyD17
Copy link

I tried to do it, and I found two main problems here:

  1. All fields that will not be returned must be Optional or have a default value. If not there will be an Exception here --- https://github.com/redis/redis-om-python/blob/main/aredis_om/model/model.py#L1473. It will not be clear for user. This can be validated in FindQuery init, but I don't have an idea, how to check it.
  2. Here is a problem with dict fields and embedded model --- it will be returned as JSON, and we will have a validation issue again.
address
Input should be a valid dictionary or object to extract fields from [type=model_attributes_type, input_value='{"pk":"01JM7319T5SP7A6YR...e":"11111","note":null}', input_type=str]

bsbodden added a commit that referenced this pull request Aug 12, 2025
- Move RETURN clause from query string to args list (fixes RediSearch syntax)
- Rename internal parameter from return_fields to projected_fields to avoid conflict with method name
- Return dictionaries instead of model instances when using field projection
- Add proper parsing for projected results that returns flat key-value pairs
- Add tests for both HashModel and JsonModel field projection
- Update validation to use model_fields instead of deprecated __fields__

This addresses all review comments from PR #633 and implements field projection correctly.
@bsbodden bsbodden force-pushed the limit_return_fields branch from 3de85c9 to 8fc78fc Compare August 12, 2025 21:18
@bsbodden bsbodden self-requested a review August 12, 2025 21:18
bsbodden added a commit that referenced this pull request Aug 12, 2025
- Move RETURN clause from query string to args list (fixes RediSearch syntax)
- Rename internal parameter from return_fields to projected_fields to avoid conflict with method name
- Return dictionaries instead of model instances when using field projection
- Add proper parsing for projected results that returns flat key-value pairs
- Add tests for both HashModel and JsonModel field projection
- Update validation to use model_fields instead of deprecated __fields__

This addresses all review comments from PR #633 and implements field projection correctly.
@bsbodden bsbodden force-pushed the limit_return_fields branch from 8fc78fc to 5b91067 Compare August 12, 2025 21:31
- Move RETURN clause from query string to args list (fixes RediSearch syntax)
- Rename internal parameter from return_fields to projected_fields to avoid conflict with method name
- Return dictionaries instead of model instances when using field projection
- Add proper parsing for projected results that returns flat key-value pairs
- Add tests for both HashModel and JsonModel field projection
- Update validation to use model_fields instead of deprecated __fields__

This addresses all review comments from PR #633 and implements field projection correctly.
@bsbodden bsbodden force-pushed the limit_return_fields branch from 5b91067 to b997389 Compare August 12, 2025 21:35
@bsbodden bsbodden removed the request for review from banker August 12, 2025 21:50
@bsbodden bsbodden self-assigned this Aug 12, 2025
Copy link
Contributor

@bsbodden bsbodden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me... now 😉

@bsbodden bsbodden requested a review from slorello89 August 12, 2025 21:53
@bsbodden bsbodden self-requested a review August 13, 2025 16:00
Copy link
Contributor

@bsbodden bsbodden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@bsbodden bsbodden requested a review from abrookins August 13, 2025 16:36
abrookins and others added 4 commits August 13, 2025 13:46
Add Django-style .values() method to return query results as dictionaries:
- .values() returns all fields as dicts
- .values('field1', 'field2') returns specific fields as dicts
- Uses Redis RETURN clause for efficient field projection
- Extensible design supports future .only() method for partial models

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <[email protected]>
@abrookins
Copy link
Collaborator

Refactored this feature to become a new only(*fields) and values(*fields) pair, following patterns from the Django ORM.

@abrookins abrookins merged commit 7e08669 into main Aug 15, 2025
11 checks passed
@abrookins abrookins deleted the limit_return_fields branch August 15, 2025 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Limit fields to return with RETURN keyword
6 participants