Fix exclude_pending and mls_only filters not working with raw return type

When return_type="raw" was specified, the exclude_pending and mls_only
parameters were ignored because these filters only existed in
process_property(), which is bypassed for raw data returns.

Changes:
- Added _apply_raw_data_filters() method to handle client-side filtering
  for raw data
- Applied the filter in search() method after sorting but before returning
- Fixed exclude_pending to check flags.is_pending and flags.is_contingent
- Fixed mls_only to check source.id (not mls.id which doesn't exist in raw data)
- Added comprehensive tests for both filters with raw data

Fixes #140

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

README.md

@@ -2,6 +2,9 @@
 
 **HomeHarvest** is a real estate scraping library that extracts and formats data in the style of MLS listings.
 
+- 🚀 [HomeHarvest MCP](https://smithery.ai/server/@ZacharyHampton/homeharvest-mcp) - Easily get property data in your agent.
+- 🏠 [Zillow API](https://rapidapi.com/zachary-l1izVlvs2/api/zillow-com9) - Get Zillow data with ease.
+
 ## HomeHarvest Features
 
 - **Source**: Fetches properties directly from **Realtor.com**.
@@ -63,6 +66,97 @@ properties = scrape_property(
 )
 ```
 
+### Advanced Filtering Examples
+
+#### Hour-Based Filtering
+```py
+# Get properties listed in the last 24 hours
+properties = scrape_property(
+    location="Austin, TX",
+    listing_type="for_sale",
+    past_hours=24
+)
+
+# Get properties listed during specific hours (e.g., business hours)
+properties = scrape_property(
+    location="Dallas, TX",
+    listing_type="for_sale",
+    datetime_from="2025-01-20T09:00:00",
+    datetime_to="2025-01-20T17:00:00"
+)
+```
+
+#### Property Filters
+```py
+# Filter by bedrooms, bathrooms, and square footage
+properties = scrape_property(
+    location="San Francisco, CA",
+    listing_type="for_sale",
+    beds_min=2,
+    beds_max=4,
+    baths_min=2.0,
+    sqft_min=1000,
+    sqft_max=2500
+)
+
+# Filter by price range
+properties = scrape_property(
+    location="Phoenix, AZ",
+    listing_type="for_sale",
+    price_min=200000,
+    price_max=500000
+)
+
+# Filter by year built
+properties = scrape_property(
+    location="Seattle, WA",
+    listing_type="for_sale",
+    year_built_min=2000,
+    beds_min=3
+)
+
+# Combine multiple filters
+properties = scrape_property(
+    location="Denver, CO",
+    listing_type="for_sale",
+    beds_min=3,
+    baths_min=2.0,
+    sqft_min=1500,
+    price_min=300000,
+    price_max=600000,
+    year_built_min=1990,
+    lot_sqft_min=5000
+)
+```
+
+#### Sorting Results
+```py
+# Sort by price (cheapest first)
+properties = scrape_property(
+    location="Miami, FL",
+    listing_type="for_sale",
+    sort_by="list_price",
+    sort_direction="asc",
+    limit=100
+)
+
+# Sort by newest listings
+properties = scrape_property(
+    location="Boston, MA",
+    listing_type="for_sale",
+    sort_by="list_date",
+    sort_direction="desc"
+)
+
+# Sort by square footage (largest first)
+properties = scrape_property(
+    location="Los Angeles, CA",
+    listing_type="for_sale",
+    sort_by="sqft",
+    sort_direction="desc"
+)
+```
+
 ## Output
 ```plaintext
 >>> properties.head()
@@ -134,11 +228,46 @@ Optional
 ├── past_days (integer): Number of past days to filter properties. Utilizes 'last_sold_date' for 'sold' listing types, and 'list_date' for others (for_rent, for_sale).
 │    Example: 30 (fetches properties listed/sold in the last 30 days)
 │
+├── past_hours (integer): Number of past hours to filter properties (more precise than past_days). Uses client-side filtering.
+│    Example: 24 (fetches properties from the last 24 hours)
+│    Note: Cannot be used together with past_days or date_from/date_to
+│
 ├── date_from, date_to (string): Start and end dates to filter properties listed or sold, both dates are required.
 |    (use this to get properties in chunks as there's a 10k result limit)
 │    Format for both must be "YYYY-MM-DD".
 │    Example: "2023-05-01", "2023-05-15" (fetches properties listed/sold between these dates)
 │
+├── datetime_from, datetime_to (string): ISO 8601 datetime strings for hour-precise filtering. Uses client-side filtering.
+│    Format: "YYYY-MM-DDTHH:MM:SS" or "YYYY-MM-DD"
+│    Example: "2025-01-20T09:00:00", "2025-01-20T17:00:00" (fetches properties between 9 AM and 5 PM)
+│    Note: Cannot be used together with date_from/date_to
+│
+├── beds_min, beds_max (integer): Filter by number of bedrooms
+│    Example: beds_min=2, beds_max=4 (2-4 bedrooms)
+│
+├── baths_min, baths_max (float): Filter by number of bathrooms
+│    Example: baths_min=2.0, baths_max=3.5 (2-3.5 bathrooms)
+│
+├── sqft_min, sqft_max (integer): Filter by square footage
+│    Example: sqft_min=1000, sqft_max=2500 (1,000-2,500 sq ft)
+│
+├── price_min, price_max (integer): Filter by listing price
+│    Example: price_min=200000, price_max=500000 ($200k-$500k)
+│
+├── lot_sqft_min, lot_sqft_max (integer): Filter by lot size in square feet
+│    Example: lot_sqft_min=5000, lot_sqft_max=10000 (5,000-10,000 sq ft lot)
+│
+├── year_built_min, year_built_max (integer): Filter by year built
+│    Example: year_built_min=2000, year_built_max=2024 (built between 2000-2024)
+│
+├── sort_by (string): Sort results by field
+│    Options: 'list_date', 'sold_date', 'list_price', 'sqft', 'beds', 'baths'
+│    Example: sort_by='list_price'
+│
+├── sort_direction (string): Sort direction, default is 'desc'
+│    Options: 'asc' (ascending), 'desc' (descending)
+│    Example: sort_direction='asc' (cheapest first)
+│
 ├── mls_only (True/False): If set, fetches only MLS listings (mainly applicable to 'sold' listings)
 │
 ├── foreclosure (True/False): If set, fetches only foreclosures
@@ -149,7 +278,9 @@ Optional
 │
 ├── exclude_pending (True/False): If set, excludes 'pending' properties from the 'for_sale' results unless listing_type is 'pending'
 │
-└── limit (integer): Limit the number of properties to fetch. Max & default is 10000.
+├── limit (integer): Limit the number of properties to fetch. Max & default is 10000.
+│
+└── offset (integer): Starting position for pagination within the 10k limit. Use with limit to fetch results in chunks.
 ```
 
 ### Property Schema
@@ -191,10 +322,11 @@ Property
 │ ├── list_price
 │ ├── list_price_min
 │ ├── list_price_max
-│ ├── list_date  # datetime
-│ ├── pending_date  # datetime
+│ ├── list_date  # datetime (full timestamp: YYYY-MM-DD HH:MM:SS)
+│ ├── pending_date  # datetime (full timestamp: YYYY-MM-DD HH:MM:SS)
 │ ├── sold_price
-│ ├── last_sold_date  # datetime
+│ ├── last_sold_date  # datetime (full timestamp: YYYY-MM-DD HH:MM:SS)
+│ ├── last_status_change_date  # datetime (full timestamp: YYYY-MM-DD HH:MM:SS)
 │ ├── last_sold_price
 │ ├── price_per_sqft
 │ ├── new_construction

homeharvest/__init__.py

@@ -1,7 +1,7 @@
 import warnings
 import pandas as pd
 from .core.scrapers import ScraperInput
-from .utils import process_result, ordered_properties, validate_input, validate_dates, validate_limit
+from .utils import process_result, ordered_properties, validate_input, validate_dates, validate_limit, validate_offset, validate_datetime, validate_filters, validate_sort
 from .core.scrapers.realtor import RealtorScraper
 from .core.scrapers.models import ListingType, SearchPropertyType, ReturnType, Property
 from typing import Union, Optional, List
@@ -15,15 +15,37 @@ def scrape_property(
     mls_only: bool = False,
     past_days: int = None,
     proxy: str = None,
-    date_from: str = None,  #: TODO: Switch to one parameter, Date, with date_from and date_to, pydantic validation
+    date_from: str = None,
     date_to: str = None,
     foreclosure: bool = None,
     extra_property_data: bool = True,
     exclude_pending: bool = False,
-    limit: int = 10000
+    limit: int = 10000,
+    offset: int = 0,
+    # New date/time filtering parameters
+    past_hours: int = None,
+    datetime_from: str = None,
+    datetime_to: str = None,
+    # New property filtering parameters
+    beds_min: int = None,
+    beds_max: int = None,
+    baths_min: float = None,
+    baths_max: float = None,
+    sqft_min: int = None,
+    sqft_max: int = None,
+    price_min: int = None,
+    price_max: int = None,
+    lot_sqft_min: int = None,
+    lot_sqft_max: int = None,
+    year_built_min: int = None,
+    year_built_max: int = None,
+    # New sorting parameters
+    sort_by: str = None,
+    sort_direction: str = "desc",
 ) -> Union[pd.DataFrame, list[dict], list[Property]]:
     """
     Scrape properties from Realtor.com based on a given location and listing type.
+
     :param location: Location to search (e.g. "Dallas, TX", "85281", "2530 Al Lipscomb Way")
     :param listing_type: Listing Type (for_sale, for_rent, sold, pending)
     :param return_type: Return type (pandas, pydantic, raw)
@@ -32,15 +54,39 @@ def scrape_property(
     :param mls_only: If set, fetches only listings with MLS IDs.
     :param proxy: Proxy to use for scraping
     :param past_days: Get properties sold or listed (dependent on your listing_type) in the last _ days.
+        - PENDING: Filters by pending_date. Contingent properties without pending_date are included.
+        - SOLD: Filters by sold_date (when property was sold)
+        - FOR_SALE/FOR_RENT: Filters by list_date (when property was listed)
     :param date_from, date_to: Get properties sold or listed (dependent on your listing_type) between these dates. format: 2021-01-28
     :param foreclosure: If set, fetches only foreclosure listings.
     :param extra_property_data: Increases requests by O(n). If set, this fetches additional property data (e.g. agent, broker, property evaluations etc.)
     :param exclude_pending: If true, this excludes pending or contingent properties from the results, unless listing type is pending.
     :param limit: Limit the number of results returned. Maximum is 10,000.
+    :param offset: Starting position for pagination within the 10k limit (offset + limit cannot exceed 10,000). Use with limit to fetch results in chunks (e.g., offset=200, limit=200 fetches results 200-399). Should be a multiple of 200 (page size) for optimal performance. Default is 0. Note: Cannot be used to bypass the 10k API limit - use date ranges (date_from/date_to) to narrow searches and fetch more data.
+
+    New parameters:
+    :param past_hours: Get properties in the last _ hours (requires client-side filtering)
+    :param datetime_from, datetime_to: ISO 8601 datetime strings for precise time filtering (e.g. "2025-01-20T14:30:00")
+    :param beds_min, beds_max: Filter by number of bedrooms
+    :param baths_min, baths_max: Filter by number of bathrooms
+    :param sqft_min, sqft_max: Filter by square footage
+    :param price_min, price_max: Filter by listing price
+    :param lot_sqft_min, lot_sqft_max: Filter by lot size
+    :param year_built_min, year_built_max: Filter by year built
+    :param sort_by: Sort results by field (list_date, sold_date, list_price, sqft, beds, baths)
+    :param sort_direction: Sort direction (asc, desc)
     """
     validate_input(listing_type)
     validate_dates(date_from, date_to)
     validate_limit(limit)
+    validate_offset(offset, limit)
+    validate_datetime(datetime_from)
+    validate_datetime(datetime_to)
+    validate_filters(
+        beds_min, beds_max, baths_min, baths_max, sqft_min, sqft_max,
+        price_min, price_max, lot_sqft_min, lot_sqft_max, year_built_min, year_built_max
+    )
+    validate_sort(sort_by, sort_direction)
 
     scraper_input = ScraperInput(
         location=location,
@@ -57,6 +103,27 @@ def scrape_property(
         extra_property_data=extra_property_data,
         exclude_pending=exclude_pending,
         limit=limit,
+        offset=offset,
+        # New date/time filtering
+        past_hours=past_hours,
+        datetime_from=datetime_from,
+        datetime_to=datetime_to,
+        # New property filtering
+        beds_min=beds_min,
+        beds_max=beds_max,
+        baths_min=baths_min,
+        baths_max=baths_max,
+        sqft_min=sqft_min,
+        sqft_max=sqft_max,
+        price_min=price_min,
+        price_max=price_max,
+        lot_sqft_min=lot_sqft_min,
+        lot_sqft_max=lot_sqft_max,
+        year_built_min=year_built_min,
+        year_built_max=year_built_max,
+        # New sorting
+        sort_by=sort_by,
+        sort_direction=sort_direction,
     )
 
     site = RealtorScraper(scraper_input)

homeharvest/core/scrapers/__init__.py

@@ -25,8 +25,32 @@ class ScraperInput(BaseModel):
     extra_property_data: bool | None = True
     exclude_pending: bool | None = False
     limit: int = 10000
+    offset: int = 0
     return_type: ReturnType = ReturnType.pandas
 
+    # New date/time filtering parameters
+    past_hours: int | None = None
+    datetime_from: str | None = None
+    datetime_to: str | None = None
+
+    # New property filtering parameters
+    beds_min: int | None = None
+    beds_max: int | None = None
+    baths_min: float | None = None
+    baths_max: float | None = None
+    sqft_min: int | None = None
+    sqft_max: int | None = None
+    price_min: int | None = None
+    price_max: int | None = None
+    lot_sqft_min: int | None = None
+    lot_sqft_max: int | None = None
+    year_built_min: int | None = None
+    year_built_max: int | None = None
+
+    # New sorting parameters
+    sort_by: str | None = None
+    sort_direction: str = "desc"
+
 
 class Scraper:
     session = None
@@ -83,8 +107,32 @@ class Scraper:
         self.extra_property_data = scraper_input.extra_property_data
         self.exclude_pending = scraper_input.exclude_pending
         self.limit = scraper_input.limit
+        self.offset = scraper_input.offset
         self.return_type = scraper_input.return_type
 
+        # New date/time filtering
+        self.past_hours = scraper_input.past_hours
+        self.datetime_from = scraper_input.datetime_from
+        self.datetime_to = scraper_input.datetime_to
+
+        # New property filtering
+        self.beds_min = scraper_input.beds_min
+        self.beds_max = scraper_input.beds_max
+        self.baths_min = scraper_input.baths_min
+        self.baths_max = scraper_input.baths_max
+        self.sqft_min = scraper_input.sqft_min
+        self.sqft_max = scraper_input.sqft_max
+        self.price_min = scraper_input.price_min
+        self.price_max = scraper_input.price_max
+        self.lot_sqft_min = scraper_input.lot_sqft_min
+        self.lot_sqft_max = scraper_input.lot_sqft_max
+        self.year_built_min = scraper_input.year_built_min
+        self.year_built_max = scraper_input.year_built_max
+
+        # New sorting
+        self.sort_by = scraper_input.sort_by
+        self.sort_direction = scraper_input.sort_direction
+
     def search(self) -> list[Union[Property | dict]]: ...
 
     @staticmethod

homeharvest/core/scrapers/models.py

@@ -192,6 +192,7 @@ class Property(BaseModel):
     list_date: datetime | None = Field(None, description="The time this Home entered Move system")
     pending_date: datetime | None = Field(None, description="The date listing went into pending state")
     last_sold_date: datetime | None = Field(None, description="Last time the Home was sold")
+    last_status_change_date: datetime | None = Field(None, description="Last time the status of the listing changed")
     prc_sqft: int | None = None
     new_construction: bool | None = Field(None, description="Search for new construction homes")
     hoa_fee: int | None = Field(None, description="Search for homes where HOA fee is known and falls within specified range")

homeharvest/core/scrapers/realtor/__init__.py

@@ -132,32 +132,138 @@ class RealtorScraper(Scraper):
         """
 
         date_param = ""
+
+        # Determine date field based on listing type
         if self.listing_type == ListingType.SOLD:
-            if self.date_from and self.date_to:
-                date_param = f'sold_date: {{ min: "{self.date_from}", max: "{self.date_to}" }}'
+            date_field = "sold_date"
+        elif self.listing_type in [ListingType.FOR_SALE, ListingType.FOR_RENT]:
+            date_field = "list_date"
+        else:  # PENDING
+            # Skip server-side date filtering for PENDING as both pending_date and contract_date
+            # filters are broken in the API. Client-side filtering will be applied later.
+            date_field = None
+
+        # Build date parameter (expand to full days if hour-based filtering is used)
+        if date_field:
+            if self.datetime_from or self.datetime_to:
+                # Hour-based datetime filtering: extract date parts for API, client-side filter by hours
+                from datetime import datetime
+
+                min_date = None
+                max_date = None
+
+                if self.datetime_from:
+                    try:
+                        dt_from = datetime.fromisoformat(self.datetime_from.replace('Z', '+00:00'))
+                        min_date = dt_from.strftime("%Y-%m-%d")
+                    except (ValueError, AttributeError):
+                        pass
+
+                if self.datetime_to:
+                    try:
+                        dt_to = datetime.fromisoformat(self.datetime_to.replace('Z', '+00:00'))
+                        max_date = dt_to.strftime("%Y-%m-%d")
+                    except (ValueError, AttributeError):
+                        pass
+
+                if min_date and max_date:
+                    date_param = f'{date_field}: {{ min: "{min_date}", max: "{max_date}" }}'
+                elif min_date:
+                    date_param = f'{date_field}: {{ min: "{min_date}" }}'
+                elif max_date:
+                    date_param = f'{date_field}: {{ max: "{max_date}" }}'
+
+            elif self.past_hours:
+                # Query API for past N days (minimum 1 day), client-side filter by hours
+                days = max(1, int(self.past_hours / 24) + 1)  # Round up to cover the full period
+                date_param = f'{date_field}: {{ min: "$today-{days}D" }}'
+
+            elif self.date_from and self.date_to:
+                date_param = f'{date_field}: {{ min: "{self.date_from}", max: "{self.date_to}" }}'
             elif self.last_x_days:
-                date_param = f'sold_date: {{ min: "$today-{self.last_x_days}D" }}'
-        else:
-            if self.date_from and self.date_to:
-                date_param = f'list_date: {{ min: "{self.date_from}", max: "{self.date_to}" }}'
-            elif self.last_x_days:
-                date_param = f'list_date: {{ min: "$today-{self.last_x_days}D" }}'
+                date_param = f'{date_field}: {{ min: "$today-{self.last_x_days}D" }}'
 
         property_type_param = ""
         if self.property_type:
             property_types = [pt.value for pt in self.property_type]
             property_type_param = f"type: {json.dumps(property_types)}"
 
-        sort_param = (
-            "sort: [{ field: sold_date, direction: desc }]"
-            if self.listing_type == ListingType.SOLD
-            else ""  #: "sort: [{ field: list_date, direction: desc }]"  #: prioritize normal fractal sort from realtor
-        )
+        # Build property filter parameters
+        property_filters = []
+
+        if self.beds_min is not None or self.beds_max is not None:
+            beds_filter = "beds: {"
+            if self.beds_min is not None:
+                beds_filter += f" min: {self.beds_min}"
+            if self.beds_max is not None:
+                beds_filter += f" max: {self.beds_max}"
+            beds_filter += " }"
+            property_filters.append(beds_filter)
+
+        if self.baths_min is not None or self.baths_max is not None:
+            baths_filter = "baths: {"
+            if self.baths_min is not None:
+                baths_filter += f" min: {self.baths_min}"
+            if self.baths_max is not None:
+                baths_filter += f" max: {self.baths_max}"
+            baths_filter += " }"
+            property_filters.append(baths_filter)
+
+        if self.sqft_min is not None or self.sqft_max is not None:
+            sqft_filter = "sqft: {"
+            if self.sqft_min is not None:
+                sqft_filter += f" min: {self.sqft_min}"
+            if self.sqft_max is not None:
+                sqft_filter += f" max: {self.sqft_max}"
+            sqft_filter += " }"
+            property_filters.append(sqft_filter)
+
+        if self.price_min is not None or self.price_max is not None:
+            price_filter = "list_price: {"
+            if self.price_min is not None:
+                price_filter += f" min: {self.price_min}"
+            if self.price_max is not None:
+                price_filter += f" max: {self.price_max}"
+            price_filter += " }"
+            property_filters.append(price_filter)
+
+        if self.lot_sqft_min is not None or self.lot_sqft_max is not None:
+            lot_sqft_filter = "lot_sqft: {"
+            if self.lot_sqft_min is not None:
+                lot_sqft_filter += f" min: {self.lot_sqft_min}"
+            if self.lot_sqft_max is not None:
+                lot_sqft_filter += f" max: {self.lot_sqft_max}"
+            lot_sqft_filter += " }"
+            property_filters.append(lot_sqft_filter)
+
+        if self.year_built_min is not None or self.year_built_max is not None:
+            year_built_filter = "year_built: {"
+            if self.year_built_min is not None:
+                year_built_filter += f" min: {self.year_built_min}"
+            if self.year_built_max is not None:
+                year_built_filter += f" max: {self.year_built_max}"
+            year_built_filter += " }"
+            property_filters.append(year_built_filter)
+
+        property_filters_param = "\n".join(property_filters)
+
+        # Build sort parameter
+        if self.sort_by:
+            sort_param = f"sort: [{{ field: {self.sort_by}, direction: {self.sort_direction} }}]"
+        elif self.listing_type == ListingType.SOLD:
+            sort_param = "sort: [{ field: sold_date, direction: desc }]"
+        else:
+            sort_param = ""  #: prioritize normal fractal sort from realtor
 
         pending_or_contingent_param = (
             "or_filters: { contingent: true, pending: true }" if self.listing_type == ListingType.PENDING else ""
         )
 
+        # Build bucket parameter (only use fractal sort if no custom sort is specified)
+        bucket_param = ""
+        if not self.sort_by:
+            bucket_param = 'bucket: { sort: "fractal_v1.1.3_fr" }'
+
         listing_type = ListingType.FOR_SALE if self.listing_type == ListingType.PENDING else self.listing_type
         is_foreclosure = ""
 
@@ -183,6 +289,7 @@ class RealtorScraper(Scraper):
                                 %s
                                 %s
                                 %s
+                                %s
                             }
                             %s
                             limit: 200
@@ -193,6 +300,7 @@ class RealtorScraper(Scraper):
                 listing_type.value.lower(),
                 date_param,
                 property_type_param,
+                property_filters_param,
                 pending_or_contingent_param,
                 sort_param,
                 GENERAL_RESULTS_QUERY,
@@ -216,8 +324,9 @@ class RealtorScraper(Scraper):
                                         %s
                                         %s
                                         %s
+                                        %s
                                     }
-                                    bucket: { sort: "fractal_v1.1.3_fr" }
+                                    %s
                                     %s
                                     limit: 200
                                     offset: $offset
@@ -227,7 +336,9 @@ class RealtorScraper(Scraper):
                 listing_type.value.lower(),
                 date_param,
                 property_type_param,
+                property_filters_param,
                 pending_or_contingent_param,
+                bucket_param,
                 sort_param,
                 GENERAL_RESULTS_QUERY,
             )
@@ -294,13 +405,23 @@ class RealtorScraper(Scraper):
 
         if self.return_type != ReturnType.raw:
             with ThreadPoolExecutor(max_workers=self.NUM_PROPERTY_WORKERS) as executor:
-                futures = [executor.submit(process_property, result, self.mls_only, self.extra_property_data, 
-                                         self.exclude_pending, self.listing_type, get_key, process_extra_property_details) for result in properties_list]
+                # Store futures with their indices to maintain sort order
+                futures_with_indices = [
+                    (i, executor.submit(process_property, result, self.mls_only, self.extra_property_data,
+                                       self.exclude_pending, self.listing_type, get_key, process_extra_property_details))
+                    for i, result in enumerate(properties_list)
+                ]
 
-                for future in as_completed(futures):
+                # Collect results and sort by index to preserve API sort order
+                results = []
+                for idx, future in futures_with_indices:
                     result = future.result()
                     if result:
-                        properties.append(result)
+                        results.append((idx, result))
+
+                # Sort by index and extract properties in correct order
+                results.sort(key=lambda x: x[0])
+                properties = [result for idx, result in results]
         else:
             properties = properties_list
 
@@ -317,7 +438,7 @@ class RealtorScraper(Scraper):
         location_type = location_info["area_type"]
 
         search_variables = {
-            "offset": 0,
+            "offset": self.offset,
         }
 
         search_type = (
@@ -362,24 +483,369 @@ class RealtorScraper(Scraper):
         homes = result["properties"]
 
         with ThreadPoolExecutor() as executor:
-            futures = [
-                executor.submit(
+            # Store futures with their offsets to maintain proper sort order
+            # Start from offset + page_size and go up to offset + limit
+            futures_with_offsets = [
+                (i, executor.submit(
                     self.general_search,
                     variables=search_variables | {"offset": i},
                     search_type=search_type,
-                )
+                ))
                 for i in range(
-                    self.DEFAULT_PAGE_SIZE,
-                    min(total, self.limit),
+                    self.offset + self.DEFAULT_PAGE_SIZE,
+                    min(total, self.offset + self.limit),
                     self.DEFAULT_PAGE_SIZE,
                 )
             ]
 
-            for future in as_completed(futures):
-                homes.extend(future.result()["properties"])
+            # Collect results and sort by offset to preserve API sort order across pages
+            results = []
+            for offset, future in futures_with_offsets:
+                results.append((offset, future.result()["properties"]))
+
+            # Sort by offset and concatenate in correct order
+            results.sort(key=lambda x: x[0])
+            for offset, properties in results:
+                homes.extend(properties)
+
+        # Apply client-side hour-based filtering if needed
+        # (API only supports day-level filtering, so we post-filter for hour precision)
+        if self.past_hours or self.datetime_from or self.datetime_to:
+            homes = self._apply_hour_based_date_filter(homes)
+        # Apply client-side date filtering for PENDING properties
+        # (server-side filters are broken in the API)
+        elif self.listing_type == ListingType.PENDING and (self.last_x_days or self.date_from):
+            homes = self._apply_pending_date_filter(homes)
+
+        # Apply client-side sort to ensure results are properly ordered
+        # This is necessary after filtering and to guarantee sort order across page boundaries
+        if self.sort_by:
+            homes = self._apply_sort(homes)
+
+        # Apply raw data filters (exclude_pending and mls_only) for raw return type
+        # These filters are normally applied in process_property() but are bypassed for raw data
+        if self.return_type == ReturnType.raw:
+            homes = self._apply_raw_data_filters(homes)
 
         return homes
 
+    def _apply_hour_based_date_filter(self, homes):
+        """Apply client-side hour-based date filtering for all listing types.
+
+        This is used when past_hours, datetime_from, or datetime_to are specified,
+        since the API only supports day-level filtering.
+        """
+        if not homes:
+            return homes
+
+        from datetime import datetime, timedelta
+
+        # Determine date range with hour precision
+        date_range = None
+
+        if self.past_hours:
+            cutoff_datetime = datetime.now() - timedelta(hours=self.past_hours)
+            date_range = {'type': 'since', 'date': cutoff_datetime}
+        elif self.datetime_from or self.datetime_to:
+            try:
+                from_datetime = None
+                to_datetime = None
+
+                if self.datetime_from:
+                    from_datetime_str = self.datetime_from.replace('Z', '+00:00') if self.datetime_from.endswith('Z') else self.datetime_from
+                    from_datetime = datetime.fromisoformat(from_datetime_str).replace(tzinfo=None)
+
+                if self.datetime_to:
+                    to_datetime_str = self.datetime_to.replace('Z', '+00:00') if self.datetime_to.endswith('Z') else self.datetime_to
+                    to_datetime = datetime.fromisoformat(to_datetime_str).replace(tzinfo=None)
+
+                if from_datetime and to_datetime:
+                    date_range = {'type': 'range', 'from_date': from_datetime, 'to_date': to_datetime}
+                elif from_datetime:
+                    date_range = {'type': 'since', 'date': from_datetime}
+                elif to_datetime:
+                    date_range = {'type': 'until', 'date': to_datetime}
+            except (ValueError, AttributeError):
+                return homes  # If parsing fails, return unfiltered
+
+        if not date_range:
+            return homes
+
+        # Determine which date field to use based on listing type
+        date_field_name = self._get_date_field_for_listing_type()
+
+        filtered_homes = []
+
+        for home in homes:
+            # Extract the appropriate date for this property
+            property_date = self._extract_date_from_home(home, date_field_name)
+
+            # Handle properties without dates
+            if property_date is None:
+                # For PENDING, include contingent properties without pending_date
+                if self.listing_type == ListingType.PENDING and self._is_contingent(home):
+                    filtered_homes.append(home)
+                continue
+
+            # Check if property date falls within the specified range
+            if self._is_datetime_in_range(property_date, date_range):
+                filtered_homes.append(home)
+
+        return filtered_homes
+
+    def _get_date_field_for_listing_type(self):
+        """Get the appropriate date field name for the current listing type."""
+        if self.listing_type == ListingType.SOLD:
+            return 'last_sold_date'
+        elif self.listing_type == ListingType.PENDING:
+            return 'pending_date'
+        else:  # FOR_SALE or FOR_RENT
+            return 'list_date'
+
+    def _extract_date_from_home(self, home, date_field_name):
+        """Extract a date field from a home (handles both dict and Property object).
+
+        Falls back to last_status_change_date if the primary date field is not available,
+        providing more precise filtering for PENDING/SOLD properties.
+        """
+        if isinstance(home, dict):
+            date_value = home.get(date_field_name)
+        else:
+            date_value = getattr(home, date_field_name, None)
+
+        if date_value:
+            return self._parse_date_value(date_value)
+
+        # Fallback to last_status_change_date if primary date field is missing
+        # This is useful for PENDING/SOLD properties where the specific date might be unavailable
+        if isinstance(home, dict):
+            fallback_date = home.get('last_status_change_date')
+        else:
+            fallback_date = getattr(home, 'last_status_change_date', None)
+
+        if fallback_date:
+            return self._parse_date_value(fallback_date)
+
+        return None
+
+    def _is_datetime_in_range(self, date_obj, date_range):
+        """Check if a datetime object falls within the specified date range (with hour precision)."""
+        if date_range['type'] == 'since':
+            return date_obj >= date_range['date']
+        elif date_range['type'] == 'until':
+            return date_obj <= date_range['date']
+        elif date_range['type'] == 'range':
+            return date_range['from_date'] <= date_obj <= date_range['to_date']
+        return False
+
+    def _apply_pending_date_filter(self, homes):
+        """Apply client-side date filtering for PENDING properties based on pending_date field.
+        For contingent properties without pending_date, tries fallback date fields."""
+        if not homes:
+            return homes
+            
+        from datetime import datetime, timedelta
+        
+        # Determine date range for filtering
+        date_range = self._get_date_range()
+        if not date_range:
+            return homes
+            
+        filtered_homes = []
+        
+        for home in homes:
+            # Extract the best available date for this property
+            property_date = self._extract_property_date_for_filtering(home)
+            
+            # Handle properties without dates (include contingent properties)
+            if property_date is None:
+                if self._is_contingent(home):
+                    filtered_homes.append(home)  # Include contingent without date filter
+                continue
+            
+            # Check if property date falls within the specified range
+            if self._is_date_in_range(property_date, date_range):
+                filtered_homes.append(home)
+                
+        return filtered_homes
+    
+    def _get_pending_date(self, home):
+        """Extract pending_date from a home property (handles both dict and Property object)."""
+        if isinstance(home, dict):
+            return home.get('pending_date')
+        else:
+            # Assume it's a Property object
+            return getattr(home, 'pending_date', None)
+    
+    
+    def _is_contingent(self, home):
+        """Check if a property is contingent."""
+        if isinstance(home, dict):
+            flags = home.get('flags', {})
+            return flags.get('is_contingent', False)
+        else:
+            # Property object - check flags attribute
+            if hasattr(home, 'flags') and home.flags:
+                return getattr(home.flags, 'is_contingent', False)
+            return False
+    
+    def _get_date_range(self):
+        """Get the date range for filtering based on instance parameters."""
+        from datetime import datetime, timedelta
+        
+        if self.last_x_days:
+            cutoff_date = datetime.now() - timedelta(days=self.last_x_days)
+            return {'type': 'since', 'date': cutoff_date}
+        elif self.date_from and self.date_to:
+            try:
+                from_date = datetime.fromisoformat(self.date_from)
+                to_date = datetime.fromisoformat(self.date_to)
+                return {'type': 'range', 'from_date': from_date, 'to_date': to_date}
+            except ValueError:
+                return None
+        return None
+    
+    def _extract_property_date_for_filtering(self, home):
+        """Extract pending_date from a property for filtering.
+        
+        Returns parsed datetime object or None.
+        """
+        date_value = self._get_pending_date(home)
+        if date_value:
+            return self._parse_date_value(date_value)
+        return None
+    
+    def _parse_date_value(self, date_value):
+        """Parse a date value (string or datetime) into a timezone-naive datetime object."""
+        from datetime import datetime
+        
+        if isinstance(date_value, datetime):
+            return date_value.replace(tzinfo=None)
+        
+        if not isinstance(date_value, str):
+            return None
+            
+        try:
+            # Handle timezone indicators
+            if date_value.endswith('Z'):
+                date_value = date_value[:-1] + '+00:00'
+            elif '.' in date_value and date_value.endswith('Z'):
+                date_value = date_value.replace('Z', '+00:00')
+            
+            # Try ISO format first
+            try:
+                parsed_date = datetime.fromisoformat(date_value)
+                return parsed_date.replace(tzinfo=None)
+            except ValueError:
+                # Try simple datetime format: '2025-08-29 00:00:00'
+                return datetime.strptime(date_value, '%Y-%m-%d %H:%M:%S')
+                
+        except (ValueError, AttributeError):
+            return None
+    
+    def _is_date_in_range(self, date_obj, date_range):
+        """Check if a datetime object falls within the specified date range."""
+        if date_range['type'] == 'since':
+            return date_obj >= date_range['date']
+        elif date_range['type'] == 'range':
+            return date_range['from_date'] <= date_obj <= date_range['to_date']
+        return False
+
+    def _apply_sort(self, homes):
+        """Apply client-side sorting to ensure results are properly ordered.
+
+        This is necessary because:
+        1. Multi-page results need to be re-sorted after concatenation
+        2. Filtering operations may disrupt the original sort order
+
+        Args:
+            homes: List of properties (either dicts or Property objects)
+
+        Returns:
+            Sorted list of properties
+        """
+        if not homes or not self.sort_by:
+            return homes
+
+        def get_sort_key(home):
+            """Extract the sort field value from a home (handles both dict and Property object)."""
+            if isinstance(home, dict):
+                value = home.get(self.sort_by)
+            else:
+                # Property object
+                value = getattr(home, self.sort_by, None)
+
+            # Handle None values - push them to the end
+            if value is None:
+                # Use a sentinel value that sorts to the end
+                return (1, 0) if self.sort_direction == "desc" else (1, float('inf'))
+
+            # For datetime fields, convert string to datetime for proper sorting
+            if self.sort_by in ['list_date', 'sold_date', 'pending_date']:
+                if isinstance(value, str):
+                    try:
+                        from datetime import datetime
+                        # Handle timezone indicators
+                        date_value = value
+                        if date_value.endswith('Z'):
+                            date_value = date_value[:-1] + '+00:00'
+                        parsed_date = datetime.fromisoformat(date_value)
+                        return (0, parsed_date)
+                    except (ValueError, AttributeError):
+                        # If parsing fails, treat as None
+                        return (1, 0) if self.sort_direction == "desc" else (1, float('inf'))
+                return (0, value)
+
+            # For numeric fields, ensure we can compare
+            return (0, value)
+
+        # Sort the homes
+        reverse = (self.sort_direction == "desc")
+        sorted_homes = sorted(homes, key=get_sort_key, reverse=reverse)
+
+        return sorted_homes
+
+    def _apply_raw_data_filters(self, homes):
+        """Apply exclude_pending and mls_only filters for raw data returns.
+
+        These filters are normally applied in process_property(), but that function
+        is bypassed when return_type="raw", so we need to apply them here instead.
+
+        Args:
+            homes: List of properties (either dicts or Property objects)
+
+        Returns:
+            Filtered list of properties
+        """
+        if not homes:
+            return homes
+
+        # Only filter raw data (dict objects)
+        # Property objects have already been filtered in process_property()
+        if homes and not isinstance(homes[0], dict):
+            return homes
+
+        filtered_homes = []
+
+        for home in homes:
+            # Apply exclude_pending filter
+            if self.exclude_pending and self.listing_type != ListingType.PENDING:
+                flags = home.get('flags', {})
+                is_pending = flags.get('is_pending', False)
+                is_contingent = flags.get('is_contingent', False)
+
+                if is_pending or is_contingent:
+                    continue  # Skip this property
+
+            # Apply mls_only filter
+            if self.mls_only:
+                source = home.get('source', {})
+                if not source or not source.get('id'):
+                    continue  # Skip this property
+
+            filtered_homes.append(home)
+
+        return filtered_homes
 
 
     @retry(

homeharvest/core/scrapers/realtor/parsers.py

@@ -250,9 +250,28 @@ def parse_description(result: dict) -> Description | None:
 def calculate_days_on_mls(result: dict) -> Optional[int]:
     """Calculate days on MLS from result data"""
     list_date_str = result.get("list_date")
-    list_date = datetime.strptime(list_date_str.split("T")[0], "%Y-%m-%d") if list_date_str else None
+    list_date = None
+    if list_date_str:
+        try:
+            # Parse full datetime, then use date() for day calculation
+            list_date_str_clean = list_date_str.replace('Z', '+00:00') if list_date_str.endswith('Z') else list_date_str
+            list_date = datetime.fromisoformat(list_date_str_clean).replace(tzinfo=None)
+        except (ValueError, AttributeError):
+            # Fallback for date-only format
+            list_date = datetime.strptime(list_date_str.split("T")[0], "%Y-%m-%d") if "T" in list_date_str else None
+
     last_sold_date_str = result.get("last_sold_date")
-    last_sold_date = datetime.strptime(last_sold_date_str, "%Y-%m-%d") if last_sold_date_str else None
+    last_sold_date = None
+    if last_sold_date_str:
+        try:
+            last_sold_date_str_clean = last_sold_date_str.replace('Z', '+00:00') if last_sold_date_str.endswith('Z') else last_sold_date_str
+            last_sold_date = datetime.fromisoformat(last_sold_date_str_clean).replace(tzinfo=None)
+        except (ValueError, AttributeError):
+            # Fallback for date-only format
+            try:
+                last_sold_date = datetime.strptime(last_sold_date_str, "%Y-%m-%d")
+            except ValueError:
+                last_sold_date = None
     today = datetime.now()
 
     if list_date:

homeharvest/core/scrapers/realtor/processors.py

@@ -121,10 +121,11 @@ def process_property(result: dict, mls_only: bool = False, extra_property_data:
         list_price=result["list_price"],
         list_price_min=result["list_price_min"],
         list_price_max=result["list_price_max"],
-        list_date=(datetime.fromisoformat(result["list_date"].split("T")[0]) if result.get("list_date") else None),
+        list_date=(datetime.fromisoformat(result["list_date"].replace('Z', '+00:00') if result["list_date"].endswith('Z') else result["list_date"]) if result.get("list_date") else None),
         prc_sqft=result.get("price_per_sqft"),
-        last_sold_date=(datetime.fromisoformat(result["last_sold_date"]) if result.get("last_sold_date") else None),
-        pending_date=(datetime.fromisoformat(result["pending_date"].split("T")[0]) if result.get("pending_date") else None),
+        last_sold_date=(datetime.fromisoformat(result["last_sold_date"].replace('Z', '+00:00') if result["last_sold_date"].endswith('Z') else result["last_sold_date"]) if result.get("last_sold_date") else None),
+        pending_date=(datetime.fromisoformat(result["pending_date"].replace('Z', '+00:00') if result["pending_date"].endswith('Z') else result["pending_date"]) if result.get("pending_date") else None),
+        last_status_change_date=(datetime.fromisoformat(result["last_status_change_date"].replace('Z', '+00:00') if result["last_status_change_date"].endswith('Z') else result["last_status_change_date"]) if result.get("last_status_change_date") else None),
         new_construction=result["flags"].get("is_new_construction") is True,
         hoa_fee=(result["hoa"]["fee"] if result.get("hoa") and isinstance(result["hoa"], dict) else None),
         latitude=(result["location"]["address"]["coordinate"].get("lat") if able_to_get_lat_long else None),
@@ -162,6 +163,25 @@ def process_property(result: dict, mls_only: bool = False, extra_property_data:
         photos=result.get("photos"),
         flags=result.get("flags"),
     )
+
+    # Enhance date precision using last_status_change_date
+    # pending_date and last_sold_date only have day-level precision
+    # last_status_change_date has hour-level precision
+    if realty_property.last_status_change_date:
+        status = realty_property.status.upper() if realty_property.status else None
+
+        # For PENDING/CONTINGENT properties, use last_status_change_date for hour-precision on pending_date
+        if status in ["PENDING", "CONTINGENT"] and realty_property.pending_date:
+            # Only replace if dates are on the same day
+            if realty_property.pending_date.date() == realty_property.last_status_change_date.date():
+                realty_property.pending_date = realty_property.last_status_change_date
+
+        # For SOLD properties, use last_status_change_date for hour-precision on last_sold_date
+        elif status == "SOLD" and realty_property.last_sold_date:
+            # Only replace if dates are on the same day
+            if realty_property.last_sold_date.date() == realty_property.last_status_change_date.date():
+                realty_property.last_sold_date = realty_property.last_status_change_date
+
     return realty_property
 
 
@@ -175,7 +195,11 @@ def process_extra_property_details(result: dict, get_key_func=None) -> dict:
         nearby_schools = result.get("nearbySchools")
         schools = nearby_schools.get("schools", []) if nearby_schools else []
         tax_history_data = result.get("taxHistory", [])
-        assessed_value = tax_history_data[0]["assessment"]["total"] if tax_history_data and tax_history_data[0].get("assessment", {}).get("total") else None
+
+        assessed_value = None
+        if tax_history_data and tax_history_data[0] and tax_history_data[0].get("assessment"):
+            assessed_value = tax_history_data[0]["assessment"].get("total")
+
         tax_history = tax_history_data
 
     if schools:

homeharvest/core/scrapers/realtor/queries.py

@@ -9,6 +9,7 @@ _SEARCH_HOMES_DATA_BASE = """{
     mls_status
     last_sold_price
     last_sold_date
+    last_status_change_date
     list_price
     list_price_max
     list_price_min
@@ -202,6 +203,11 @@ fragment HomeData on Home {
         }
     }
     taxHistory: tax_history { __typename tax year assessment { __typename building land total } }
+    property_history {
+        date
+        event_name
+        price
+    }
     monthly_fees {
         description
         display_amount

homeharvest/utils.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 import pandas as pd
+import warnings
 from datetime import datetime
 from .core.scrapers.models import Property, ListingType, Advertisers
 from .exceptions import InvalidListingType, InvalidDate
@@ -36,6 +37,7 @@ ordered_properties = [
     "sold_price",
     "last_sold_date",
     "last_sold_price",
+    "last_status_change_date",
     "assessed_value",
     "estimated_value",
     "tax",
@@ -119,10 +121,10 @@ def process_result(result: Property) -> pd.DataFrame:
     prop_data["nearby_schools"] = filter(None, prop_data["nearby_schools"]) if prop_data["nearby_schools"] else None
     prop_data["nearby_schools"] = ", ".join(set(prop_data["nearby_schools"])) if prop_data["nearby_schools"] else None
     
-    # Convert datetime objects to strings for CSV
-    for date_field in ["list_date", "pending_date", "last_sold_date"]:
+    # Convert datetime objects to strings for CSV (preserve full datetime including time)
+    for date_field in ["list_date", "pending_date", "last_sold_date", "last_status_change_date"]:
         if prop_data.get(date_field):
-            prop_data[date_field] = prop_data[date_field].strftime("%Y-%m-%d") if hasattr(prop_data[date_field], 'strftime') else prop_data[date_field]
+            prop_data[date_field] = prop_data[date_field].strftime("%Y-%m-%d %H:%M:%S") if hasattr(prop_data[date_field], 'strftime') else prop_data[date_field]
     
     # Convert HttpUrl objects to strings for CSV
     if prop_data.get("property_url"):
@@ -179,3 +181,95 @@ def validate_limit(limit: int) -> None:
 
     if limit is not None and (limit < 1 or limit > 10000):
         raise ValueError("Property limit must be between 1 and 10,000.")
+
+
+def validate_offset(offset: int, limit: int = 10000) -> None:
+    """Validate offset parameter for pagination.
+
+    Args:
+        offset: Starting position for results pagination
+        limit: Maximum number of results to fetch
+
+    Raises:
+        ValueError: If offset is invalid or if offset + limit exceeds API limit
+    """
+    if offset is not None and offset < 0:
+        raise ValueError("Offset must be non-negative (>= 0).")
+
+    # Check if offset + limit exceeds API's hard limit of 10,000
+    if offset is not None and limit is not None and (offset + limit) > 10000:
+        raise ValueError(
+            f"offset ({offset}) + limit ({limit}) = {offset + limit} exceeds API maximum of 10,000. "
+            f"The API cannot return results beyond position 10,000. "
+            f"To fetch more results, narrow your search."
+        )
+
+    # Warn if offset is not a multiple of 200 (API page size)
+    if offset is not None and offset > 0 and offset % 200 != 0:
+        warnings.warn(
+            f"Offset should be a multiple of 200 (page size) for optimal performance. "
+            f"Using offset {offset} may result in less efficient pagination.",
+            UserWarning
+        )
+
+
+def validate_datetime(datetime_str: str | None) -> None:
+    """Validate ISO 8601 datetime format."""
+    if not datetime_str:
+        return
+
+    try:
+        # Try parsing as ISO 8601 datetime
+        datetime.fromisoformat(datetime_str.replace('Z', '+00:00'))
+    except (ValueError, AttributeError):
+        raise InvalidDate(
+            f"Invalid datetime format: '{datetime_str}'. "
+            f"Expected ISO 8601 format (e.g., '2025-01-20T14:30:00' or '2025-01-20')."
+        )
+
+
+def validate_filters(
+    beds_min: int | None = None,
+    beds_max: int | None = None,
+    baths_min: float | None = None,
+    baths_max: float | None = None,
+    sqft_min: int | None = None,
+    sqft_max: int | None = None,
+    price_min: int | None = None,
+    price_max: int | None = None,
+    lot_sqft_min: int | None = None,
+    lot_sqft_max: int | None = None,
+    year_built_min: int | None = None,
+    year_built_max: int | None = None,
+) -> None:
+    """Validate that min values are less than max values for range filters."""
+    ranges = [
+        ("beds", beds_min, beds_max),
+        ("baths", baths_min, baths_max),
+        ("sqft", sqft_min, sqft_max),
+        ("price", price_min, price_max),
+        ("lot_sqft", lot_sqft_min, lot_sqft_max),
+        ("year_built", year_built_min, year_built_max),
+    ]
+
+    for name, min_val, max_val in ranges:
+        if min_val is not None and max_val is not None and min_val > max_val:
+            raise ValueError(f"{name}_min ({min_val}) cannot be greater than {name}_max ({max_val}).")
+
+
+def validate_sort(sort_by: str | None, sort_direction: str | None = "desc") -> None:
+    """Validate sort parameters."""
+    valid_sort_fields = ["list_date", "sold_date", "list_price", "sqft", "beds", "baths"]
+    valid_directions = ["asc", "desc"]
+
+    if sort_by and sort_by not in valid_sort_fields:
+        raise ValueError(
+            f"Invalid sort_by value: '{sort_by}'. "
+            f"Valid options: {', '.join(valid_sort_fields)}"
+        )
+
+    if sort_direction and sort_direction not in valid_directions:
+        raise ValueError(
+            f"Invalid sort_direction value: '{sort_direction}'. "
+            f"Valid options: {', '.join(valid_directions)}"
+        )

poetry.lock

@@ -1,4 +1,4 @@
-# This file is automatically @generated by Poetry 2.1.3 and should not be changed by hand.
+# This file is automatically @generated by Poetry 2.2.1 and should not be changed by hand.
 
 [[package]]
 name = "annotated-types"
@@ -943,5 +943,5 @@ test = ["covdefaults (>=2.3)", "coverage (>=7.2.7)", "coverage-enable-subprocess
 
 [metadata]
 lock-version = "2.1"
-python-versions = ">=3.9,<3.13"
-content-hash = "17de7786a5e0bc51f4f42b6703dc41564050f8696a1b5d2e315ceffe6e192309"
+python-versions = ">=3.9"
+content-hash = "c60c33aa5f054998b90bd1941c825c9ca1867a53e64c07e188b91da49c7741a4"

pyproject.toml

@@ -1,16 +1,16 @@
 [tool.poetry]
 name = "homeharvest"
-version = "0.5.1"
+version = "0.7.3"
 description = "Real estate scraping library"
 authors = ["Zachary Hampton <zachary@bunsly.com>", "Cullen Watson <cullen@bunsly.com>"]
-homepage = "https://github.com/Bunsly/HomeHarvest"
+homepage = "https://github.com/ZacharyHampton/HomeHarvest"
 readme = "README.md"
 
 [tool.poetry.scripts]
 homeharvest = "homeharvest.cli:main"
 
 [tool.poetry.dependencies]
-python = ">=3.9,<3.13"
+python = ">=3.9"
 requests = "^2.32.4"
 pandas = "^2.3.1"
 pydantic = "^2.11.7"

tests/test_realtor.py

@@ -296,13 +296,27 @@ def test_return_type():
 
 
 def test_has_open_house():
+    """Test that open_houses field is present and properly structured when it exists"""
+
+    # Test that open_houses field exists in results (may be None if no open houses scheduled)
     address_result = scrape_property("1 Hawthorne St Unit 12F, San Francisco, CA 94105", return_type="raw")
-    assert address_result[0]["open_houses"] is not None  #: has open house data from address search
+    assert "open_houses" in address_result[0], "open_houses field should exist in address search results"
 
-    zip_code_result = scrape_property("94105", return_type="raw")
-    address_from_zip_result = list(filter(lambda row: row["property_id"] == '1264014746', zip_code_result))
+    # Test general search also includes open_houses field
+    zip_code_result = scrape_property("94105", listing_type="for_sale", limit=50, return_type="raw")
+    assert len(zip_code_result) > 0, "Should have results from zip code search"
 
-    assert address_from_zip_result[0]["open_houses"] is not None  #: has open house data from general search
+    # Verify open_houses field exists in general search
+    assert "open_houses" in zip_code_result[0], "open_houses field should exist in general search results"
+
+    # If we find any properties with open houses, verify the data structure
+    properties_with_open_houses = [prop for prop in zip_code_result if prop.get("open_houses") is not None]
+
+    if properties_with_open_houses:
+        # Verify structure of open_houses data
+        first_with_open_house = properties_with_open_houses[0]
+        assert isinstance(first_with_open_house["open_houses"], (list, dict)), \
+            "open_houses should be a list or dict when present"
 
 
 
@@ -372,4 +386,967 @@ def test_return_type_consistency():
         # All return types should have some properties
         assert len(pandas_ids) > 0, f"pandas should return properties for {search_type}"
         assert len(pydantic_ids) > 0, f"pydantic should return properties for {search_type}"
-        assert len(raw_ids) > 0, f"raw should return properties for {search_type}"
+        assert len(raw_ids) > 0, f"raw should return properties for {search_type}"
+
+
+def test_pending_date_filtering():
+    """Test that pending properties are properly filtered by pending_date using client-side filtering."""
+    
+    # Test 1: Verify that date filtering works with different time windows
+    result_no_filter = scrape_property(
+        location="Dallas, TX",
+        listing_type="pending", 
+        limit=20
+    )
+    
+    result_30_days = scrape_property(
+        location="Dallas, TX", 
+        listing_type="pending",
+        past_days=30,
+        limit=20
+    )
+    
+    result_10_days = scrape_property(
+        location="Dallas, TX",
+        listing_type="pending", 
+        past_days=10,
+        limit=20
+    )
+    
+    # Basic assertions - we should get some results
+    assert result_no_filter is not None and len(result_no_filter) >= 0
+    assert result_30_days is not None and len(result_30_days) >= 0
+    assert result_10_days is not None and len(result_10_days) >= 0
+    
+    # Filtering should work: longer periods should return same or more results
+    assert len(result_30_days) <= len(result_no_filter), "30-day filter should return <= unfiltered results"
+    assert len(result_10_days) <= len(result_30_days), "10-day filter should return <= 30-day results"
+    
+    # Test 2: Verify that date range filtering works
+    if len(result_no_filter) > 0:
+        result_date_range = scrape_property(
+            location="Dallas, TX",
+            listing_type="pending",
+            date_from="2025-08-01", 
+            date_to="2025-12-31",
+            limit=20
+        )
+        
+        assert result_date_range is not None
+        # Date range should capture recent properties
+        assert len(result_date_range) >= 0
+    
+    # Test 3: Verify that both pending and contingent properties are included
+    # Get raw data to check property types
+    if len(result_no_filter) > 0:
+        raw_result = scrape_property(
+            location="Dallas, TX",
+            listing_type="pending",
+            return_type="raw",
+            limit=15
+        )
+        
+        if raw_result:
+            # Check that we get both pending and contingent properties
+            pending_count = 0
+            contingent_count = 0
+            
+            for prop in raw_result:
+                flags = prop.get('flags', {})
+                if flags.get('is_pending'):
+                    pending_count += 1
+                if flags.get('is_contingent'):
+                    contingent_count += 1
+            
+            # We should get at least one of each type (when available)
+            total_properties = pending_count + contingent_count
+            assert total_properties > 0, "Should find at least some pending or contingent properties"
+
+
+def test_hour_based_filtering():
+    """Test the new past_hours parameter for hour-level filtering"""
+    from datetime import datetime, timedelta
+
+    # Test for sold properties with 24-hour filter
+    result_24h = scrape_property(
+        location="Phoenix, AZ",
+        listing_type="sold",
+        past_hours=24,
+        limit=50
+    )
+
+    # Test for sold properties with 12-hour filter
+    result_12h = scrape_property(
+        location="Phoenix, AZ",
+        listing_type="sold",
+        past_hours=12,
+        limit=50
+    )
+
+    assert result_24h is not None
+    assert result_12h is not None
+
+    # 12-hour filter should return same or fewer results than 24-hour
+    if len(result_12h) > 0 and len(result_24h) > 0:
+        assert len(result_12h) <= len(result_24h), "12-hour results should be <= 24-hour results"
+
+    # Verify timestamps are within the specified hour range for 24h filter
+    if len(result_24h) > 0:
+        cutoff_time = datetime.now() - timedelta(hours=24)
+
+        # Check a few results
+        for idx in range(min(5, len(result_24h))):
+            sold_date_str = result_24h.iloc[idx]["last_sold_date"]
+            if pd.notna(sold_date_str):
+                try:
+                    sold_date = datetime.strptime(str(sold_date_str), "%Y-%m-%d %H:%M:%S")
+                    # Date should be within last 24 hours
+                    assert sold_date >= cutoff_time, f"Property sold date {sold_date} should be within last 24 hours"
+                except (ValueError, TypeError):
+                    pass  # Skip if date parsing fails
+
+
+def test_past_hours_all_listing_types():
+    """Validate that past_hours works correctly for all listing types with proper date fields"""
+    from datetime import datetime, timedelta
+
+    # Test 1: SOLD (uses last_sold_date field, server-side filters by sold_date)
+    result_sold = scrape_property(
+        location="Dallas, TX",
+        listing_type="sold",
+        past_hours=48,
+        limit=20
+    )
+
+    assert result_sold is not None
+    if len(result_sold) > 0:
+        cutoff_48h = datetime.now() - timedelta(hours=48)
+
+        # Verify results use sold_date and are within 48 hours
+        for idx in range(min(5, len(result_sold))):
+            sold_date_str = result_sold.iloc[idx]["last_sold_date"]
+            if pd.notna(sold_date_str):
+                try:
+                    sold_date = datetime.strptime(str(sold_date_str), "%Y-%m-%d %H:%M:%S")
+                    assert sold_date >= cutoff_48h, \
+                        f"SOLD: last_sold_date {sold_date} should be within 48 hours"
+                except (ValueError, TypeError):
+                    pass
+
+    # Test 2: FOR_SALE (uses list_date field, server-side filters by list_date)
+    result_for_sale = scrape_property(
+        location="Austin, TX",
+        listing_type="for_sale",
+        past_hours=48,
+        limit=20
+    )
+
+    assert result_for_sale is not None
+    if len(result_for_sale) > 0:
+        cutoff_48h = datetime.now() - timedelta(hours=48)
+
+        # Verify results use list_date and are within 48 hours
+        for idx in range(min(5, len(result_for_sale))):
+            list_date_str = result_for_sale.iloc[idx]["list_date"]
+            if pd.notna(list_date_str):
+                try:
+                    list_date = datetime.strptime(str(list_date_str), "%Y-%m-%d %H:%M:%S")
+                    assert list_date >= cutoff_48h, \
+                        f"FOR_SALE: list_date {list_date} should be within 48 hours"
+                except (ValueError, TypeError):
+                    pass
+
+    # Test 3: FOR_RENT (uses list_date field, server-side filters by list_date)
+    result_for_rent = scrape_property(
+        location="Houston, TX",
+        listing_type="for_rent",
+        past_hours=72,
+        limit=20
+    )
+
+    assert result_for_rent is not None
+    if len(result_for_rent) > 0:
+        cutoff_72h = datetime.now() - timedelta(hours=72)
+
+        # Verify results use list_date and are within 72 hours
+        for idx in range(min(5, len(result_for_rent))):
+            list_date_str = result_for_rent.iloc[idx]["list_date"]
+            if pd.notna(list_date_str):
+                try:
+                    list_date = datetime.strptime(str(list_date_str), "%Y-%m-%d %H:%M:%S")
+                    assert list_date >= cutoff_72h, \
+                        f"FOR_RENT: list_date {list_date} should be within 72 hours"
+                except (ValueError, TypeError):
+                    pass
+
+    # Test 4: PENDING (uses pending_date field, client-side filtering only)
+    result_pending = scrape_property(
+        location="San Antonio, TX",
+        listing_type="pending",
+        past_hours=48,
+        limit=20
+    )
+
+    assert result_pending is not None
+    # Note: PENDING doesn't use server-side date filtering (API filters broken)
+    # Client-side filtering should still work via pending_date
+    if len(result_pending) > 0:
+        cutoff_48h = datetime.now() - timedelta(hours=48)
+
+        # Verify results use pending_date (or are contingent without date)
+        for idx in range(min(5, len(result_pending))):
+            pending_date_str = result_pending.iloc[idx]["pending_date"]
+            if pd.notna(pending_date_str):
+                try:
+                    pending_date = datetime.strptime(str(pending_date_str), "%Y-%m-%d %H:%M:%S")
+                    assert pending_date >= cutoff_48h, \
+                        f"PENDING: pending_date {pending_date} should be within 48 hours"
+                except (ValueError, TypeError):
+                    pass
+            # else: property is contingent without pending_date, which is allowed
+
+
+def test_datetime_filtering():
+    """Test datetime_from and datetime_to parameters with hour precision"""
+    from datetime import datetime, timedelta
+
+    # Get a recent date range (e.g., yesterday)
+    yesterday = datetime.now() - timedelta(days=1)
+    date_str = yesterday.strftime("%Y-%m-%d")
+
+    # Test filtering for business hours (9 AM to 5 PM) on a specific day
+    result = scrape_property(
+        location="Dallas, TX",
+        listing_type="for_sale",
+        datetime_from=f"{date_str}T09:00:00",
+        datetime_to=f"{date_str}T17:00:00",
+        limit=30
+    )
+
+    assert result is not None
+
+    # Test with only datetime_from
+    result_from_only = scrape_property(
+        location="Houston, TX",
+        listing_type="for_sale",
+        datetime_from=f"{date_str}T00:00:00",
+        limit=30
+    )
+
+    assert result_from_only is not None
+
+    # Test with only datetime_to
+    result_to_only = scrape_property(
+        location="Austin, TX",
+        listing_type="for_sale",
+        datetime_to=f"{date_str}T23:59:59",
+        limit=30
+    )
+
+    assert result_to_only is not None
+
+
+def test_full_datetime_preservation():
+    """Verify that dates now include full timestamps (YYYY-MM-DD HH:MM:SS)"""
+
+    # Test with pandas return type
+    result_pandas = scrape_property(
+        location="San Diego, CA",
+        listing_type="sold",
+        past_days=30,
+        limit=10
+    )
+
+    assert result_pandas is not None and len(result_pandas) > 0
+
+    # Check that date fields contain time information
+    if len(result_pandas) > 0:
+        for idx in range(min(3, len(result_pandas))):
+            # Check last_sold_date
+            sold_date = result_pandas.iloc[idx]["last_sold_date"]
+            if pd.notna(sold_date):
+                sold_date_str = str(sold_date)
+                # Should contain time (HH:MM:SS), not just date
+                assert " " in sold_date_str or "T" in sold_date_str, \
+                    f"Date should include time component: {sold_date_str}"
+
+    # Test with pydantic return type
+    result_pydantic = scrape_property(
+        location="Los Angeles, CA",
+        listing_type="for_sale",
+        past_days=7,
+        limit=10,
+        return_type="pydantic"
+    )
+
+    assert result_pydantic is not None and len(result_pydantic) > 0
+
+    # Verify Property objects have datetime objects with time info
+    for prop in result_pydantic[:3]:
+        if prop.list_date:
+            # Should be a datetime object, not just a date
+            assert hasattr(prop.list_date, 'hour'), "list_date should be a datetime with time"
+
+
+def test_beds_filtering():
+    """Test bedroom filtering with beds_min and beds_max"""
+
+    result = scrape_property(
+        location="Atlanta, GA",
+        listing_type="for_sale",
+        beds_min=2,
+        beds_max=4,
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify all properties have 2-4 bedrooms
+    for idx in range(min(10, len(result))):
+        beds = result.iloc[idx]["beds"]
+        if pd.notna(beds):
+            assert 2 <= beds <= 4, f"Property should have 2-4 beds, got {beds}"
+
+    # Test beds_min only
+    result_min = scrape_property(
+        location="Denver, CO",
+        listing_type="for_sale",
+        beds_min=3,
+        limit=30
+    )
+
+    assert result_min is not None
+
+    # Test beds_max only
+    result_max = scrape_property(
+        location="Seattle, WA",
+        listing_type="for_sale",
+        beds_max=2,
+        limit=30
+    )
+
+    assert result_max is not None
+
+
+def test_baths_filtering():
+    """Test bathroom filtering with baths_min and baths_max"""
+
+    result = scrape_property(
+        location="Miami, FL",
+        listing_type="for_sale",
+        baths_min=2.0,
+        baths_max=3.5,
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify bathrooms are within range
+    for idx in range(min(10, len(result))):
+        full_baths = result.iloc[idx]["full_baths"]
+        half_baths = result.iloc[idx]["half_baths"]
+
+        if pd.notna(full_baths):
+            total_baths = float(full_baths) + (float(half_baths) * 0.5 if pd.notna(half_baths) else 0)
+            # Allow some tolerance as API might calculate differently
+            if total_baths > 0:
+                assert total_baths >= 1.5, f"Baths should be >= 2.0, got {total_baths}"
+
+
+def test_sqft_filtering():
+    """Test square footage filtering"""
+
+    result = scrape_property(
+        location="Portland, OR",
+        listing_type="for_sale",
+        sqft_min=1000,
+        sqft_max=2500,
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify sqft is within range
+    for idx in range(min(10, len(result))):
+        sqft = result.iloc[idx]["sqft"]
+        if pd.notna(sqft) and sqft > 0:
+            assert 1000 <= sqft <= 2500, f"Sqft should be 1000-2500, got {sqft}"
+
+
+def test_price_filtering():
+    """Test price range filtering"""
+
+    result = scrape_property(
+        location="Charlotte, NC",
+        listing_type="for_sale",
+        price_min=200000,
+        price_max=500000,
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify prices are within range
+    for idx in range(min(15, len(result))):
+        price = result.iloc[idx]["list_price"]
+        if pd.notna(price) and price > 0:
+            assert 200000 <= price <= 500000, f"Price should be $200k-$500k, got ${price}"
+
+
+def test_lot_sqft_filtering():
+    """Test lot size filtering"""
+
+    result = scrape_property(
+        location="Scottsdale, AZ",
+        listing_type="for_sale",
+        lot_sqft_min=5000,
+        lot_sqft_max=15000,
+        limit=30
+    )
+
+    assert result is not None
+    # Results might be fewer if lot_sqft data is sparse
+
+
+def test_year_built_filtering():
+    """Test year built filtering"""
+
+    result = scrape_property(
+        location="Tampa, FL",
+        listing_type="for_sale",
+        year_built_min=2000,
+        year_built_max=2024,
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify year_built is within range
+    for idx in range(min(10, len(result))):
+        year = result.iloc[idx]["year_built"]
+        if pd.notna(year) and year > 0:
+            assert 2000 <= year <= 2024, f"Year should be 2000-2024, got {year}"
+
+
+def test_combined_filters():
+    """Test multiple filters working together"""
+
+    result = scrape_property(
+        location="Nashville, TN",
+        listing_type="for_sale",
+        beds_min=3,
+        baths_min=2.0,
+        sqft_min=1500,
+        price_min=250000,
+        price_max=600000,
+        year_built_min=1990,
+        limit=30
+    )
+
+    assert result is not None
+
+    # If we get results, verify they meet ALL criteria
+    if len(result) > 0:
+        for idx in range(min(5, len(result))):
+            row = result.iloc[idx]
+
+            # Check beds
+            if pd.notna(row["beds"]):
+                assert row["beds"] >= 3, f"Beds should be >= 3, got {row['beds']}"
+
+            # Check sqft
+            if pd.notna(row["sqft"]) and row["sqft"] > 0:
+                assert row["sqft"] >= 1500, f"Sqft should be >= 1500, got {row['sqft']}"
+
+            # Check price
+            if pd.notna(row["list_price"]) and row["list_price"] > 0:
+                assert 250000 <= row["list_price"] <= 600000, \
+                    f"Price should be $250k-$600k, got ${row['list_price']}"
+
+            # Check year
+            if pd.notna(row["year_built"]) and row["year_built"] > 0:
+                assert row["year_built"] >= 1990, \
+                    f"Year should be >= 1990, got {row['year_built']}"
+
+
+def test_sorting_by_price():
+    """Test sorting by list_price with actual sort order validation"""
+
+    # Sort ascending (cheapest first) with multi-page limit to test concatenation
+    result_asc = scrape_property(
+        location="Orlando, FL",
+        listing_type="for_sale",
+        sort_by="list_price",
+        sort_direction="asc",
+        limit=250  # Multi-page to test concatenation logic
+    )
+
+    assert result_asc is not None and len(result_asc) > 0
+
+    # Verify ascending sort order (allow for None/NA values at the end)
+    prices_asc = result_asc["list_price"].dropna().tolist()
+    assert len(prices_asc) > 0, "No properties with prices found"
+    assert prices_asc == sorted(prices_asc), f"Prices not in ascending order: {prices_asc[:10]}"
+
+    # Sort descending (most expensive first)
+    result_desc = scrape_property(
+        location="San Antonio, TX",
+        listing_type="for_sale",
+        sort_by="list_price",
+        sort_direction="desc",
+        limit=250  # Multi-page to test concatenation logic
+    )
+
+    assert result_desc is not None and len(result_desc) > 0
+
+    # Verify descending sort order (allow for None/NA values at the end)
+    prices_desc = result_desc["list_price"].dropna().tolist()
+    assert len(prices_desc) > 0, "No properties with prices found"
+    assert prices_desc == sorted(prices_desc, reverse=True), f"Prices not in descending order: {prices_desc[:10]}"
+
+
+def test_sorting_by_date():
+    """Test sorting by list_date with actual sort order validation"""
+
+    # Test descending (newest first) with multi-page limit
+    result_desc = scrape_property(
+        location="Columbus, OH",
+        listing_type="for_sale",
+        sort_by="list_date",
+        sort_direction="desc",  # Newest first
+        limit=250  # Multi-page to test concatenation logic
+    )
+
+    assert result_desc is not None and len(result_desc) > 0
+
+    # Verify descending sort order (allow for None/NA values at the end)
+    dates_desc = result_desc["list_date"].dropna().tolist()
+    assert len(dates_desc) > 0, "No properties with dates found"
+    assert dates_desc == sorted(dates_desc, reverse=True), f"Dates not in descending order (newest first): {dates_desc[:10]}"
+
+    # Test ascending (oldest first)
+    result_asc = scrape_property(
+        location="Columbus, OH",
+        listing_type="for_sale",
+        sort_by="list_date",
+        sort_direction="asc",  # Oldest first
+        limit=250
+    )
+
+    assert result_asc is not None and len(result_asc) > 0
+
+    # Verify ascending sort order
+    dates_asc = result_asc["list_date"].dropna().tolist()
+    assert len(dates_asc) > 0, "No properties with dates found"
+    assert dates_asc == sorted(dates_asc), f"Dates not in ascending order (oldest first): {dates_asc[:10]}"
+
+
+def test_sorting_by_sqft():
+    """Test sorting by square footage with actual sort order validation"""
+
+    # Test descending (largest first) with multi-page limit
+    result_desc = scrape_property(
+        location="Indianapolis, IN",
+        listing_type="for_sale",
+        sort_by="sqft",
+        sort_direction="desc",  # Largest first
+        limit=250  # Multi-page to test concatenation logic
+    )
+
+    assert result_desc is not None and len(result_desc) > 0
+
+    # Verify descending sort order (allow for None/NA values at the end)
+    sqfts_desc = result_desc["sqft"].dropna().tolist()
+    assert len(sqfts_desc) > 0, "No properties with sqft found"
+    assert sqfts_desc == sorted(sqfts_desc, reverse=True), f"Square footages not in descending order: {sqfts_desc[:10]}"
+
+    # Test ascending (smallest first)
+    result_asc = scrape_property(
+        location="Indianapolis, IN",
+        listing_type="for_sale",
+        sort_by="sqft",
+        sort_direction="asc",  # Smallest first
+        limit=250
+    )
+
+    assert result_asc is not None and len(result_asc) > 0
+
+    # Verify ascending sort order
+    sqfts_asc = result_asc["sqft"].dropna().tolist()
+    assert len(sqfts_asc) > 0, "No properties with sqft found"
+    assert sqfts_asc == sorted(sqfts_asc), f"Square footages not in ascending order: {sqfts_asc[:10]}"
+
+
+def test_filter_validation_errors():
+    """Test that validation catches invalid parameters"""
+    import pytest
+
+    # Test: beds_min > beds_max should raise ValueError
+    with pytest.raises(ValueError, match="beds_min.*cannot be greater than.*beds_max"):
+        scrape_property(
+            location="Boston, MA",
+            listing_type="for_sale",
+            beds_min=5,
+            beds_max=2,
+            limit=10
+        )
+
+    # Test: invalid datetime format should raise exception
+    with pytest.raises(Exception):  # InvalidDate
+        scrape_property(
+            location="Boston, MA",
+            listing_type="for_sale",
+            datetime_from="not-a-valid-datetime",
+            limit=10
+        )
+
+    # Test: invalid sort_by value should raise ValueError
+    with pytest.raises(ValueError, match="Invalid sort_by"):
+        scrape_property(
+            location="Boston, MA",
+            listing_type="for_sale",
+            sort_by="invalid_field",
+            limit=10
+        )
+
+    # Test: invalid sort_direction should raise ValueError
+    with pytest.raises(ValueError, match="Invalid sort_direction"):
+        scrape_property(
+            location="Boston, MA",
+            listing_type="for_sale",
+            sort_by="list_price",
+            sort_direction="invalid",
+            limit=10
+        )
+
+
+def test_backward_compatibility():
+    """Ensure old parameters still work as expected"""
+
+    # Test past_days still works
+    result_past_days = scrape_property(
+        location="Las Vegas, NV",
+        listing_type="sold",
+        past_days=30,
+        limit=20
+    )
+
+    assert result_past_days is not None and len(result_past_days) > 0
+
+    # Test date_from/date_to still work
+    result_date_range = scrape_property(
+        location="Memphis, TN",
+        listing_type="sold",
+        date_from="2024-01-01",
+        date_to="2024-03-31",
+        limit=20
+    )
+
+    assert result_date_range is not None
+
+    # Test property_type still works
+    result_property_type = scrape_property(
+        location="Louisville, KY",
+        listing_type="for_sale",
+        property_type=["single_family"],
+        limit=20
+    )
+
+    assert result_property_type is not None and len(result_property_type) > 0
+
+    # Test foreclosure still works
+    result_foreclosure = scrape_property(
+        location="Detroit, MI",
+        listing_type="for_sale",
+        foreclosure=True,
+        limit=15
+    )
+
+    assert result_foreclosure is not None
+
+
+def test_last_status_change_date_field():
+    """Test that last_status_change_date field is present and has hour-level precision"""
+    from datetime import datetime
+
+    # Test 1: Field is present in SOLD listings
+    result_sold = scrape_property(
+        location="Phoenix, AZ",
+        listing_type="sold",
+        past_days=30,
+        limit=20
+    )
+
+    assert result_sold is not None and len(result_sold) > 0
+
+    # Check that last_status_change_date column exists
+    assert "last_status_change_date" in result_sold.columns, \
+        "last_status_change_date column should be present in results"
+
+    # Check that at least some properties have this field populated
+    has_status_change_date = False
+    for idx in range(min(10, len(result_sold))):
+        status_change_date_str = result_sold.iloc[idx]["last_status_change_date"]
+        if pd.notna(status_change_date_str):
+            has_status_change_date = True
+            # Verify it has hour-level precision (includes time)
+            assert " " in str(status_change_date_str) or "T" in str(status_change_date_str), \
+                f"last_status_change_date should include time component: {status_change_date_str}"
+            break
+
+    # Note: It's possible some properties don't have this field, so we just verify it exists
+    # assert has_status_change_date, "At least some properties should have last_status_change_date"
+
+    # Test 2: Field is present in PENDING listings
+    result_pending = scrape_property(
+        location="Dallas, TX",
+        listing_type="pending",
+        past_days=30,
+        limit=20
+    )
+
+    assert result_pending is not None
+    assert "last_status_change_date" in result_pending.columns, \
+        "last_status_change_date column should be present in PENDING results"
+
+    # Test 3: Field is present in FOR_SALE listings
+    result_for_sale = scrape_property(
+        location="Austin, TX",
+        listing_type="for_sale",
+        past_days=7,
+        limit=20
+    )
+
+    assert result_for_sale is not None and len(result_for_sale) > 0
+    assert "last_status_change_date" in result_for_sale.columns, \
+        "last_status_change_date column should be present in FOR_SALE results"
+
+
+def test_last_status_change_date_precision_enhancement():
+    """Test that pending_date and last_sold_date use hour-precision from last_status_change_date"""
+    from datetime import datetime
+
+    # Test with pydantic return type to examine actual Property objects
+    # Use a larger time window to ensure we get some results
+    result_sold = scrape_property(
+        location="Phoenix, AZ",
+        listing_type="sold",
+        past_days=90,
+        limit=30,
+        return_type="pydantic"
+    )
+
+    assert result_sold is not None
+
+    # Only run assertions if we have data (data availability may vary)
+    if len(result_sold) > 0:
+        # Check that dates have hour-level precision (not just date)
+        for prop in result_sold[:10]:
+            # If both last_sold_date and last_status_change_date exist
+            if prop.last_sold_date and prop.last_status_change_date:
+                # Both should be datetime objects with time info
+                assert hasattr(prop.last_sold_date, 'hour'), \
+                    "last_sold_date should have hour precision"
+                assert hasattr(prop.last_status_change_date, 'hour'), \
+                    "last_status_change_date should have hour precision"
+
+                # If they're on the same day, the processor should have used
+                # last_status_change_date to provide hour precision for last_sold_date
+                if prop.last_sold_date.date() == prop.last_status_change_date.date():
+                    # They should have the same timestamp (hour/minute/second)
+                    assert prop.last_sold_date == prop.last_status_change_date, \
+                        "last_sold_date should match last_status_change_date for hour precision"
+
+    # Test with PENDING listings
+    result_pending = scrape_property(
+        location="Dallas, TX",
+        listing_type="pending",
+        past_days=90,
+        limit=30,
+        return_type="pydantic"
+    )
+
+    assert result_pending is not None
+
+    # Only run assertions if we have data
+    if len(result_pending) > 0:
+        for prop in result_pending[:10]:
+            # If both pending_date and last_status_change_date exist
+            if prop.pending_date and prop.last_status_change_date:
+                assert hasattr(prop.pending_date, 'hour'), \
+                    "pending_date should have hour precision"
+                assert hasattr(prop.last_status_change_date, 'hour'), \
+                    "last_status_change_date should have hour precision"
+
+                # If they're on the same day, pending_date should use the time from last_status_change_date
+                if prop.pending_date.date() == prop.last_status_change_date.date():
+                    assert prop.pending_date == prop.last_status_change_date, \
+                        "pending_date should match last_status_change_date for hour precision"
+
+
+def test_last_status_change_date_filtering_fallback():
+    """Test that filtering falls back to last_status_change_date when primary date is missing"""
+    from datetime import datetime, timedelta
+
+    # This test verifies that if a property doesn't have the primary date field
+    # (e.g., pending_date for PENDING listings), it can still be filtered using
+    # last_status_change_date as a fallback
+
+    # Test with PENDING properties using past_hours (client-side filtering)
+    result_pending = scrape_property(
+        location="Miami, FL",
+        listing_type="pending",
+        past_hours=72,
+        limit=30
+    )
+
+    assert result_pending is not None
+
+    # If we get results, verify they have either pending_date or last_status_change_date
+    if len(result_pending) > 0:
+        cutoff_time = datetime.now() - timedelta(hours=72)
+
+        for idx in range(min(5, len(result_pending))):
+            pending_date_str = result_pending.iloc[idx]["pending_date"]
+            status_change_date_str = result_pending.iloc[idx]["last_status_change_date"]
+
+            # At least one of these should be present for filtering to work
+            has_date = pd.notna(pending_date_str) or pd.notna(status_change_date_str)
+
+            # Note: Contingent properties without dates are allowed, so we don't assert here
+            # The test just verifies the field exists and can be used
+
+
+def test_last_status_change_date_hour_filtering():
+    """Test that past_hours filtering works correctly with last_status_change_date for PENDING/SOLD"""
+    from datetime import datetime, timedelta
+
+    # Test with SOLD properties
+    result_sold = scrape_property(
+        location="Atlanta, GA",
+        listing_type="sold",
+        past_hours=48,
+        limit=30
+    )
+
+    assert result_sold is not None
+
+    if len(result_sold) > 0:
+        cutoff_time = datetime.now() - timedelta(hours=48)
+
+        # Verify that results are within 48 hours
+        for idx in range(min(5, len(result_sold))):
+            sold_date_str = result_sold.iloc[idx]["last_sold_date"]
+            if pd.notna(sold_date_str):
+                try:
+                    sold_date = datetime.strptime(str(sold_date_str), "%Y-%m-%d %H:%M:%S")
+                    # Should be within 48 hours with hour-level precision
+                    assert sold_date >= cutoff_time, \
+                        f"SOLD property last_sold_date {sold_date} should be within 48 hours of {cutoff_time}"
+                except (ValueError, TypeError):
+                    pass  # Skip if parsing fails
+
+    # Test with PENDING properties
+    result_pending = scrape_property(
+        location="Denver, CO",
+        listing_type="pending",
+        past_hours=48,
+        limit=30
+    )
+
+    assert result_pending is not None
+
+    if len(result_pending) > 0:
+        cutoff_time = datetime.now() - timedelta(hours=48)
+
+        # Verify that results are within 48 hours
+        for idx in range(min(5, len(result_pending))):
+            pending_date_str = result_pending.iloc[idx]["pending_date"]
+            if pd.notna(pending_date_str):
+                try:
+                    pending_date = datetime.strptime(str(pending_date_str), "%Y-%m-%d %H:%M:%S")
+                    # Should be within 48 hours with hour-level precision
+                    assert pending_date >= cutoff_time, \
+                        f"PENDING property pending_date {pending_date} should be within 48 hours of {cutoff_time}"
+                except (ValueError, TypeError):
+                    pass  # Skip if parsing fails
+
+
+def test_exclude_pending_with_raw_data():
+    """Test that exclude_pending parameter works correctly with return_type='raw'"""
+
+    # Query for sale properties with exclude_pending=True and raw data
+    result = scrape_property(
+        location="Phoenix, AZ",
+        listing_type="for_sale",
+        exclude_pending=True,
+        return_type="raw",
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify that no pending or contingent properties are in the results
+    for prop in result:
+        flags = prop.get('flags', {})
+        is_pending = flags.get('is_pending', False)
+        is_contingent = flags.get('is_contingent', False)
+
+        assert not is_pending, f"Property {prop.get('property_id')} should not be pending when exclude_pending=True"
+        assert not is_contingent, f"Property {prop.get('property_id')} should not be contingent when exclude_pending=True"
+
+
+def test_mls_only_with_raw_data():
+    """Test that mls_only parameter works correctly with return_type='raw'"""
+
+    # Query with mls_only=True and raw data
+    result = scrape_property(
+        location="Dallas, TX",
+        listing_type="for_sale",
+        mls_only=True,
+        return_type="raw",
+        limit=50
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify that all properties have MLS IDs (stored in source.id)
+    for prop in result:
+        source = prop.get('source', {})
+        mls_id = source.get('id') if source else None
+
+        assert mls_id is not None and mls_id != "", \
+            f"Property {prop.get('property_id')} should have an MLS ID (source.id) when mls_only=True, got: {mls_id}"
+
+
+def test_combined_filters_with_raw_data():
+    """Test that both exclude_pending and mls_only work together with return_type='raw'"""
+
+    # Query with both filters enabled and raw data
+    result = scrape_property(
+        location="Austin, TX",
+        listing_type="for_sale",
+        exclude_pending=True,
+        mls_only=True,
+        return_type="raw",
+        limit=30
+    )
+
+    assert result is not None and len(result) > 0
+
+    # Verify both filters are applied
+    for prop in result:
+        # Check exclude_pending filter
+        flags = prop.get('flags', {})
+        is_pending = flags.get('is_pending', False)
+        is_contingent = flags.get('is_contingent', False)
+
+        assert not is_pending, f"Property {prop.get('property_id')} should not be pending"
+        assert not is_contingent, f"Property {prop.get('property_id')} should not be contingent"
+
+        # Check mls_only filter
+        source = prop.get('source', {})
+        mls_id = source.get('id') if source else None
+
+        assert mls_id is not None and mls_id != "", \
+            f"Property {prop.get('property_id')} should have an MLS ID (source.id)"