Commit Graph

56 Commits

Author SHA1 Message Date
Zachary Hampton
a6fe0d2675 Add last_update_date filtering and improve time interface DX
Part A: Add last_update_date filtering (client-side)
- Add updated_since parameter (accepts datetime object or ISO string)
- Add updated_in_past_hours parameter (accepts int or timedelta)
- Implement _apply_last_update_date_filter() method for client-side filtering
- Add mutual exclusion validation for updated_* parameters

Part B: Improve time interface DX
- Accept datetime/timedelta objects for datetime_from, datetime_to
- Accept timedelta objects for past_hours, past_days
- Add type conversion helper functions in utils.py
- Improve validation error messages with specific examples
- Update validate_datetime to accept datetime objects

Helper functions added:
- convert_to_datetime_string() - Converts datetime objects to ISO strings
- extract_timedelta_hours() - Extracts hours from timedelta objects
- extract_timedelta_days() - Extracts days from timedelta objects
- validate_last_update_filters() - Validates last_update_date parameters

All changes are backward compatible - existing string/int parameters still work.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 12:00:15 -08:00
Zachary Hampton
3a0e91b876 Add flexible listing_type support and last_update_date field
- Add support for str, list[str], and None as listing_type values
  - Single string: maintains backward compatibility (e.g., "for_sale")
  - List of strings: returns properties matching ANY status (OR logic)
  - None: returns all property types (omits status filter)

- Expand ListingType enum with all GraphQL HomeStatus values
  - Add OFF_MARKET, NEW_COMMUNITY, OTHER, READY_TO_BUILD

- Add last_update_date field support
  - Add to GraphQL query, Property model, and processors
  - Add to sort validation and datetime field sorting
  - Field description: "Last time the home was updated"

- Update GraphQL query construction to support status arrays
  - Single type: status: for_sale
  - Multiple types: status: [for_sale, sold]
  - None: omit status parameter entirely

- Update validation logic to handle new parameter types

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-11 11:28:35 -08:00
Zachary Hampton
21b6ba44f4 Add pagination offset support for API queries
Implements offset parameter to enable pagination within the 10k API limit. Users can now fetch results in chunks (e.g., offset=200, limit=200 for results 200-399). Includes validation to ensure offset + limit doesn't exceed API maximum. Also fixes multi-page result sorting to preserve correct order across page boundaries.

Fixes #139

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-05 10:57:01 -08:00
Zachary Hampton
18815e4207 Add comprehensive filtering system with hour-based datetime and property filters
This major enhancement addresses user needs for more precise filtering and introduces
powerful new capabilities for property searches:

Key Features:
- Hour-based date filtering (past_hours, datetime_from/to with ISO 8601 support)
- Server-side property filters (beds, baths, sqft, price, lot_sqft, year_built)
- Sorting support (list_date, sold_date, list_price, sqft, beds, baths)
- Full timestamp preservation (YYYY-MM-DD HH:MM:SS instead of date-only)
- Comprehensive validation with helpful error messages

Technical Changes:
- Preserve full datetime precision in processors.py and parsers.py
- Implement client-side hour-based post-filtering for all listing types
- Add server-side GraphQL filters for property characteristics
- Generalize filtering to work across SOLD, PENDING, FOR_SALE, FOR_RENT
- Add 15 comprehensive tests covering all new features
- Maintain full backward compatibility with existing parameters

Fixes #113 (sorting support)

Version bump to 0.7.0 reflects significant new functionality while maintaining
backward compatibility.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-20 14:21:12 -07:00
Zachary Hampton
75c245cde7 implement client-side pending_date filtering for PENDING properties
- Fix PENDING properties to filter by pending_date instead of list_date
- Add client-side filtering for PENDING as server-side pending_date filter is broken
- Include contingent properties without pending_date for comprehensive results
- Enhance documentation to clarify past_days behavior per listing type
- Add property_history field to GraphQL queries for future enhancements
- Add comprehensive test for pending date filtering functionality
- Optimize filtering logic with helper methods for better maintainability

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-08 16:36:48 -07:00
Alexandre Shofstall
62b6726d42 Fix syntax of __init__ line 24 2025-07-03 19:20:49 +02:00
Alexandre Shofstall
941d1081f7 Fix typing syntax for Python 3.9 compatibility in __init__.py 2025-07-03 18:11:18 +02:00
zachary
8a5683fe79 - return type parameter
- optimized get extra fields with query clustering
2025-04-12 17:55:52 -07:00
Cullen Watson
8e04f6b117 enh: property type (#102) 2024-11-03 17:23:07 -06:00
Zachary Hampton
3f44744d61 - primary photo bug fix
- limit parameter
2024-07-15 07:19:57 -07:00
Zachary Hampton
93e6778a48 - exclude_pending parameter 2024-05-31 22:17:29 -07:00
Zachary Hampton
ec036bb989 - optimizations & updated realtor headers 2024-05-20 12:13:30 -07:00
Zachary Hampton
803fd618e9 - data cleaning & CONDOP bug fixes 2024-05-12 21:12:12 -07:00
Zachary Hampton
c3e24a4ce0 - extra_property_details parameter
- updated docs
- classified exception
2024-05-02 09:04:49 -07:00
Zachary Hampton
46985dcee4 - various data quality fixes (including #70) 2024-05-02 08:48:53 -07:00
robertomr100
ab6a0e3b6e Add foreclosure parameter (#55) 2024-03-03 11:45:28 -06:00
Cullen Watson
4edad901c5 [enh] date_to and date_from 2023-11-03 18:40:34 -05:00
Zachary Hampton
6ede8622cc - pending listing support
- removal of pending_or_contingent param
2023-10-05 11:43:00 -07:00
Zachary Hampton
a9225b532f - rename days variable 2023-10-04 21:35:14 -07:00
Zachary Hampton
8a5f0dc2c9 - pending or contingent support 2023-10-04 18:25:01 -07:00
Zachary Hampton
de692faae2 - rename last_x_days
- docstrings for scrape_property
2023-10-04 18:06:06 -07:00
Cullen Watson
c4870677c2 [enh]: make last_x_days generic
add mls_only
make radius generic
2023-10-04 10:11:53 -05:00
Cullen Watson
51bde20c3c [chore]: clean up 2023-10-04 08:58:55 -05:00
Zachary Hampton
29664e4eee - cullen merge 2023-10-03 22:21:16 -07:00
Zachary Hampton
088088ae51 - last x days param 2023-10-03 15:05:17 -07:00
Zachary Hampton
40bbf76db1 - realtor radius 2023-10-02 13:58:47 -07:00
Zachary Hampton
1f1ca8068f - realtor.com default 2023-10-02 10:28:13 -07:00
Zachary Hampton
01c53f9399 - redfin bug fix
- add recent features for issues
2023-09-28 15:19:43 -07:00
Cullen Watson
644f16b25b feat: keep duplicates flag 2023-09-20 20:24:18 -05:00
Cullen Watson
f6054e8746 fix: simplify fields 2023-09-19 21:13:20 -05:00
Cullen Watson
fd9cdea499 feat: proxy support 2023-09-19 15:43:24 -05:00
Cullen Watson
49d27943c4 add cli 2023-09-19 15:01:39 -05:00
Cullen Watson
dba1c03081 feat(redfin): add sold listing_type 2023-09-19 12:27:13 -05:00
Cullen Watson
1fc2d8c549 feat(redfin): rental support 2023-09-19 11:58:20 -05:00
Cullen Watson
62e3321277 fix(zillow): test case 2023-09-18 22:59:49 -05:00
Zachary Hampton
3ec47c5b6a - invalid test cases
- redfin and realtor bug fixes
- dupe check bug fix
2023-09-18 20:28:03 -07:00
Cullen Watson
2eec389838 docs: add logo 2023-09-18 21:02:12 -05:00
Cullen Watson
4726764482 refactor: merge master 2023-09-18 17:46:05 -05:00
Cullen Watson
ca260fd2b4 fix: filter dup on street, unit, city 2023-09-18 17:42:16 -05:00
Zachary Hampton
94e5b090da - refactor 2023-09-18 15:22:43 -07:00
Cullen Watson
588689c230 fix: normalize unit num 2023-09-18 17:04:34 -05:00
Cullen Watson
c7a4bfd5e4 feat: run all 3 sites with one scrape_property() call 2023-09-18 16:18:22 -05:00
Cullen Watson
869d7e7c51 refator(realtor): fit to updated models 2023-09-18 15:43:44 -05:00
Cullen Watson
ffd3ce6aed reactor(redfin) 2023-09-18 14:36:18 -05:00
Cullen Watson
dc8c15959f fix: use zillow backend ep 2023-09-18 13:38:17 -05:00
Zachary Hampton
6b02394e95 - scrape_property docstring 2023-09-18 08:37:07 -07:00
Zachary Hampton
ba249ca20d - redfin buildings support 2023-09-18 08:26:35 -07:00
Zachary Hampton
ba9fe806a7 - finished realtor 2023-09-18 08:16:59 -07:00
Cullen Watson
905cfcae2c refactor: scrape_property() 2023-09-17 18:52:34 -05:00
Cullen Watson
3697b7cf2d feat: add pandas 2023-09-17 18:30:37 -05:00