## Performance Optimizations
### Auto-Apply Optimal Sort
- Auto-apply `sort_by="last_update_date"` when using `updated_since` or `updated_in_past_hours`
- Auto-apply `sort_by="pending_date"` when using PENDING listings with date filters
- Ensures API returns properties in chronological order for efficient filtering
- Users can still override by specifying different `sort_by`
### Early Termination
- Pre-check page 1 before launching parallel pagination
- If last property is outside time window, stop pagination immediately
- Avoids 95%+ of unnecessary API calls for narrow time windows
- Only applies when conditions guarantee correctness (date sort + time filter)
## Impact
- 10x faster for narrow time windows (2-3 seconds vs 30+ seconds)
- Fixes inefficiency where 10,000 properties fetched to return 10 matches
- Maintains backward compatibility - falls back when optimization unavailable
## Changes
- homeharvest/__init__.py: Auto-sort logic for time filters
- homeharvest/core/scrapers/realtor/__init__.py: `_should_fetch_more_pages()` method + early termination in pagination
- tests/test_realtor.py: Tests for optimization behavior
- README.md: Updated parameters documentation with all 8 listing types
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix validate_dates() to allow date_from or date_to individually
- Update test_datetime_filtering to use date_from/date_to instead of datetime_from/datetime_to
- Fix test_return_type zip code (66642 -> 85281) to ensure rental availability
- Rewrite test_realtor_without_extra_details assertions to check specific fields
- Add empty DataFrame check in test_last_status_change_date_field
All 48 tests now passing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Simplified the time filtering interface by consolidating datetime_from/datetime_to
into date_from/date_to with automatic precision detection.
Changes:
- Remove datetime_from and datetime_to parameters (confusing to have both)
- Update date_from/date_to to accept multiple formats:
- Date strings: "2025-01-20" (day precision)
- Datetime strings: "2025-01-20T14:30:00" (hour precision)
- date objects: date(2025, 1, 20) (day precision)
- datetime objects: datetime(2025, 1, 20, 9, 0) (hour precision)
- Add detect_precision_and_convert() helper to automatically detect precision
- Add date_from_precision and date_to_precision fields to track precision level
- Update filtering logic to use precision fields instead of separate parameters
- Update README to remove datetime_from/datetime_to examples
- Update validation to accept ISO datetime strings
Benefits:
- Single, intuitive parameter name (date_from/date_to)
- Automatic precision detection based on input format
- Reduced API surface area and cognitive load
- More Pythonic - accept multiple input types
All changes are backward compatible for existing date_from/date_to string usage.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Add support for str, list[str], and None as listing_type values
- Single string: maintains backward compatibility (e.g., "for_sale")
- List of strings: returns properties matching ANY status (OR logic)
- None: returns all property types (omits status filter)
- Expand ListingType enum with all GraphQL HomeStatus values
- Add OFF_MARKET, NEW_COMMUNITY, OTHER, READY_TO_BUILD
- Add last_update_date field support
- Add to GraphQL query, Property model, and processors
- Add to sort validation and datetime field sorting
- Field description: "Last time the home was updated"
- Update GraphQL query construction to support status arrays
- Single type: status: for_sale
- Multiple types: status: [for_sale, sold]
- None: omit status parameter entirely
- Update validation logic to handle new parameter types
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
When return_type="raw" was specified, the exclude_pending and mls_only
parameters were ignored because these filters only existed in
process_property(), which is bypassed for raw data returns.
Changes:
- Added _apply_raw_data_filters() method to handle client-side filtering
for raw data
- Applied the filter in search() method after sorting but before returning
- Fixed exclude_pending to check flags.is_pending and flags.is_contingent
- Fixed mls_only to check source.id (not mls.id which doesn't exist in raw data)
- Added comprehensive tests for both filters with raw data
Fixes#140🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Implements offset parameter to enable pagination within the 10k API limit. Users can now fetch results in chunks (e.g., offset=200, limit=200 for results 200-399). Includes validation to ensure offset + limit doesn't exceed API maximum. Also fixes multi-page result sorting to preserve correct order across page boundaries.
Fixes#139🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Enhances pending_date and last_sold_date with hour-level precision by introducing the last_status_change_date field. This allows for more accurate filtering of PENDING and SOLD properties when using past_hours parameter. Includes comprehensive tests and version bump to 0.7.1.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
This major enhancement addresses user needs for more precise filtering and introduces
powerful new capabilities for property searches:
Key Features:
- Hour-based date filtering (past_hours, datetime_from/to with ISO 8601 support)
- Server-side property filters (beds, baths, sqft, price, lot_sqft, year_built)
- Sorting support (list_date, sold_date, list_price, sqft, beds, baths)
- Full timestamp preservation (YYYY-MM-DD HH:MM:SS instead of date-only)
- Comprehensive validation with helpful error messages
Technical Changes:
- Preserve full datetime precision in processors.py and parsers.py
- Implement client-side hour-based post-filtering for all listing types
- Add server-side GraphQL filters for property characteristics
- Generalize filtering to work across SOLD, PENDING, FOR_SALE, FOR_RENT
- Add 15 comprehensive tests covering all new features
- Maintain full backward compatibility with existing parameters
Fixes#113 (sorting support)
Version bump to 0.7.0 reflects significant new functionality while maintaining
backward compatibility.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
- Fix PENDING properties to filter by pending_date instead of list_date
- Add client-side filtering for PENDING as server-side pending_date filter is broken
- Include contingent properties without pending_date for comprehensive results
- Enhance documentation to clarify past_days behavior per listing type
- Add property_history field to GraphQL queries for future enhancements
- Add comprehensive test for pending date filtering functionality
- Optimize filtering logic with helper methods for better maintainability
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
