Merge pull request #44 from Bunsly/fix_postal_search

fix postal search to search just by zip

README.md

@@ -2,13 +2,9 @@
 
 **HomeHarvest** is a simple, yet comprehensive, real estate scraping library that extracts and formats data in the style of MLS listings.
 
-[![Try with Replit](https://replit.com/badge?caption=Try%20with%20Replit)](https://replit.com/@ZacharyHampton/HomeHarvestDemo)
-
 **Not technical?** Try out the web scraping tool on our site at [tryhomeharvest.com](https://tryhomeharvest.com).
 
-*Looking to build a data-focused software product?* **[Book a call](https://calendly.com/zachary-products/15min)** *to work with us.*
+*Looking to build a data-focused software product?* **[Book a call](https://bunsly.com)** *to work with us.*
-
-Check out another project we wrote: ***[JobSpy](https://github.com/cullenwatson/JobSpy)** – a Python package for job scraping*
 
 ## HomeHarvest Features
 
@@ -20,7 +16,7 @@ Check out another project we wrote: ***[JobSpy](https://github.com/cullenwatson/
   - **CLI**: For users who prefer command-line operations.
 
 
-[Video Guide for HomeHarvest](https://youtu.be/JnV7eR2Ve2o) - _updated for release v0.2.7_
+[Video Guide for HomeHarvest](https://youtu.be/J1qgNPgmSLI) - _updated for release v0.3.4_
 
 ![homeharvest](https://github.com/ZacharyHampton/HomeHarvest/assets/78247585/b3d5d727-e67b-4a9f-85d8-1e65fd18620a)
 
@@ -46,7 +42,11 @@ filename = f"HomeHarvest_{current_timestamp}.csv"
 properties = scrape_property(
   location="San Diego, CA",
   listing_type="sold",  # or (for_sale, for_rent, pending)
-  past_days=30,  # sold in last 30 days - listed in last x days if (for_sale, for_rent)
+  past_days=30,  # sold in last 30 days - listed in last 30 days if (for_sale, for_rent)
+  
+  # date_from="2023-05-01", # alternative to past_days 
+  # date_to="2023-05-28", 
+  
   # mls_only=True,  # only fetch MLS listings
   # proxy="http://user:pass@host:port"  # use a proxy to change your IP address
 )
@@ -57,34 +57,6 @@ properties.to_csv(filename, index=False)
 print(properties.head())
 ```
 
-### CLI 
-
-```
-usage: homeharvest [-l {for_sale,for_rent,sold}] [-o {excel,csv}] [-f FILENAME] [-p PROXY] [-d DAYS] [-r RADIUS] [-m] [-c] location
-                                                                                                                             
-Home Harvest Property Scraper                                                                                                 
-                                                                                                                             
-positional arguments:                                                                                                         
-  location              Location to scrape (e.g., San Francisco, CA)                                                          
-                                                                                                                             
-options:                                                                                                                      
-  -l {for_sale,for_rent,sold,pending}, --listing_type {for_sale,for_rent,sold,pending}                                                        
-                        Listing type to scrape                                                                                
-  -o {excel,csv}, --output {excel,csv}                                                                                        
-                        Output format                                                                                         
-  -f FILENAME, --filename FILENAME                                                                                            
-                        Name of the output file (without extension)                                                           
-  -p PROXY, --proxy PROXY                                                                                                     
-                        Proxy to use for scraping                                                                             
-  -d DAYS, --days DAYS  Sold/listed in last _ days filter.                                                                           
-  -r RADIUS, --radius RADIUS                                                                                                  
-                        Get comparable properties within _ (e.g., 0.0) miles. Only applicable for individual addresses.        
-  -m, --mls_only        If set, fetches only MLS listings.                                                                    
-```
-```bash
-homeharvest "San Francisco, CA" -l for_rent -o excel -f HomeHarvest
-```
-
 
 ## Output
 ```plaintext
@@ -115,11 +87,46 @@ Optional
 ├── past_days (integer): Number of past days to filter properties. Utilizes 'last_sold_date' for 'sold' listing types, and 'list_date' for others (for_rent, for_sale).
 │    Example: 30 (fetches properties listed/sold in the last 30 days)
 │
+├── date_from, date_to (string): Start and end dates to filter properties listed or sold, both dates are required.
+|    (use this to get properties in chunks as there's a 10k result limit)
+│    Format for both must be "YYYY-MM-DD". 
+│    Example: "2023-05-01", "2023-05-15" (fetches properties listed/sold between these dates)
+│
 ├── mls_only (True/False): If set, fetches only MLS listings (mainly applicable to 'sold' listings)
 │
 └── proxy (string): In format 'http://user:pass@host:port'
 
+
 ```
+
+### CLI 
+
+```
+usage: homeharvest [-l {for_sale,for_rent,sold}] [-o {excel,csv}] [-f FILENAME] [-p PROXY] [-d DAYS] [-r RADIUS] [-m] [-c] location
+                                                                                                                             
+Home Harvest Property Scraper                                                                                                 
+                                                                                                                             
+positional arguments:                                                                                                         
+  location              Location to scrape (e.g., San Francisco, CA)                                                          
+                                                                                                                             
+options:                                                                                                                      
+  -l {for_sale,for_rent,sold,pending}, --listing_type {for_sale,for_rent,sold,pending}                                                        
+                        Listing type to scrape                                                                                
+  -o {excel,csv}, --output {excel,csv}                                                                                        
+                        Output format                                                                                         
+  -f FILENAME, --filename FILENAME                                                                                            
+                        Name of the output file (without extension)                                                           
+  -p PROXY, --proxy PROXY                                                                                                     
+                        Proxy to use for scraping                                                                             
+  -d DAYS, --days DAYS  Sold/listed in last _ days filter.                                                                           
+  -r RADIUS, --radius RADIUS                                                                                                  
+                        Get comparable properties within _ (e.g., 0.0) miles. Only applicable for individual addresses.        
+  -m, --mls_only        If set, fetches only MLS listings.                                                                    
+```
+```bash
+homeharvest "San Francisco, CA" -l for_rent -o excel -f HomeHarvest
+```
+
 ### Property Schema
 ```plaintext
 Property
@@ -147,6 +154,7 @@ Property
 │ └── lot_sqft
 
 ├── Property Listing Details:
+│ ├── days_on_mls
 │ ├── list_price
 │ ├── list_date
 │ ├── sold_price
@@ -166,7 +174,7 @@ Property
 The following exceptions may be raised when using HomeHarvest:
 
 - `InvalidListingType` - valid options: `for_sale`, `for_rent`, `sold`
-- `NoResultsFound` - no properties found from your search
+- `InvalidDate` - date_from or date_to is not in the format YYYY-MM-DD
   
   
 ## Frequently Asked Questions
@@ -181,7 +189,7 @@ The following exceptions may be raised when using HomeHarvest:
 **A:** This indicates that you have been blocked by Realtor.com for sending too many requests. We recommend:
 
 - Waiting a few seconds between requests.
-- Trying a VPN or useing a proxy as a parameter to scrape_property() to change your IP address.
+- Trying a VPN or using a proxy as a parameter to scrape_property() to change your IP address.
 
 ---
 

examples/HomeHarvest_Demo.ipynb

@@ -4,7 +4,9 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "cb48903e-5021-49fe-9688-45cd0bc05d0f",
-   "metadata": {},
+   "metadata": {
+    "is_executing": true
+   },
    "outputs": [],
    "source": [
     "from homeharvest import scrape_property\n",
@@ -84,10 +86,34 @@
    "outputs": [],
    "source": [
     "# check sold properties\n",
-    "scrape_property(\n",
+    "properties = scrape_property(\n",
     "    location=\"90210\",\n",
-    "    listing_type=\"sold\"\n",
+    "    listing_type=\"sold\",\n",
-    ")"
+    "    past_days=10\n",
+    ")\n",
+    "display(properties)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "628c1ce2",
+   "metadata": {
+    "collapsed": false,
+    "is_executing": true,
+    "jupyter": {
+     "outputs_hidden": false
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# display clickable URLs\n",
+    "from IPython.display import display, HTML\n",
+    "properties['property_url'] = '<a href=\"' + properties['property_url'] + '\" target=\"_blank\">' + properties['property_url'] + '</a>'\n",
+    "\n",
+    "html = properties.to_html(escape=False)\n",
+    "truncate_width = f'<style>.dataframe td {{ max-width: 200px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }}</style>{html}'\n",
+    "display(HTML(truncate_width))"
    ]
   }
  ],

homeharvest/__init__.py

@@ -1,10 +1,9 @@
 import warnings
 import pandas as pd
 from .core.scrapers import ScraperInput
-from .utils import process_result, ordered_properties, validate_input
+from .utils import process_result, ordered_properties, validate_input, validate_dates
 from .core.scrapers.realtor import RealtorScraper
 from .core.scrapers.models import ListingType
-from .exceptions import InvalidListingType, NoResultsFound
 
 
 def scrape_property(
@@ -14,6 +13,8 @@ def scrape_property(
     mls_only: bool = False,
     past_days: int = None,
     proxy: str = None,
+    date_from: str = None,
+    date_to: str = None,
 ) -> pd.DataFrame:
     """
     Scrape properties from Realtor.com based on a given location and listing type.
@@ -22,9 +23,11 @@ def scrape_property(
     :param radius: Get properties within _ (e.g. 1.0) miles. Only applicable for individual addresses.
     :param mls_only: If set, fetches only listings with MLS IDs.
     :param past_days: Get properties sold or listed (dependent on your listing_type) in the last _ days.
+    :param date_from, date_to: Get properties sold or listed (dependent on your listing_type) between these dates. format: 2021-01-28
     :param proxy: Proxy to use for scraping
     """
     validate_input(listing_type)
+    validate_dates(date_from, date_to)
 
     scraper_input = ScraperInput(
         location=location,
@@ -33,6 +36,8 @@ def scrape_property(
         radius=radius,
         mls_only=mls_only,
         last_x_days=past_days,
+        date_from=date_from,
+        date_to=date_to,
     )
 
     site = RealtorScraper(scraper_input)
@@ -40,7 +45,7 @@ def scrape_property(
 
     properties_dfs = [process_result(result) for result in results]
     if not properties_dfs:
-        raise NoResultsFound("no results found for the query")
+        return pd.DataFrame()
 
     with warnings.catch_warnings():
         warnings.simplefilter("ignore", category=FutureWarning)

homeharvest/core/scrapers/__init__.py

@@ -11,6 +11,8 @@ class ScraperInput:
     mls_only: bool | None = None
     proxy: str | None = None
     last_x_days: int | None = None
+    date_from: str | None = None
+    date_to: str | None = None
 
 
 class Scraper:
@@ -36,6 +38,8 @@ class Scraper:
         self.radius = scraper_input.radius
         self.last_x_days = scraper_input.last_x_days
         self.mls_only = scraper_input.mls_only
+        self.date_from = scraper_input.date_from
+        self.date_to = scraper_input.date_to
 
     def search(self) -> list[Property]:
         ...

homeharvest/core/scrapers/models.py

@@ -34,6 +34,8 @@ class Address:
 
 @dataclass
 class Description:
+    primary_photo: str | None = None
+    alt_photos: list[str] | None = None
     style: str | None = None
     beds: int | None = None
     baths_full: int | None = None
@@ -59,6 +61,7 @@ class Property:
     last_sold_date: str | None = None
     prc_sqft: int | None = None
     hoa_fee: int | None = None
+    days_on_mls: int | None = None
     description: Description | None = None
 
     latitude: float | None = None

homeharvest/core/scrapers/realtor/__init__.py

@@ -4,11 +4,11 @@ homeharvest.realtor.__init__
 
 This module implements the scraper for realtor.com
 """
+from datetime import datetime
 from typing import Dict, Union, Optional
 from concurrent.futures import ThreadPoolExecutor, as_completed
 
 from .. import Scraper
-from ....exceptions import NoResultsFound
 from ..models import Property, Address, ListingType, Description
 
 
@@ -37,7 +37,7 @@ class RealtorScraper(Scraper):
         result = response_json["autocomplete"]
 
         if not result:
-            raise NoResultsFound("No results found for location: " + self.location)
+            return None
 
         return result[0]
 
@@ -49,6 +49,7 @@ class RealtorScraper(Scraper):
                             listing_id
                         }
                         address {
+                            street_direction
                             street_number
                             street_name
                             street_suffix
@@ -83,6 +84,12 @@ class RealtorScraper(Scraper):
                             garage
                             permalink
                         }
+                        primary_photo {
+                            href
+                        }
+                        photos {
+                            href
+                        }
                     }
                 }"""
 
@@ -104,11 +111,29 @@ class RealtorScraper(Scraper):
         )
 
         able_to_get_lat_long = (
-            property_info
+                property_info
-            and property_info.get("address")
+                and property_info.get("address")
-            and property_info["address"].get("location")
+                and property_info["address"].get("location")
-            and property_info["address"]["location"].get("coordinate")
+                and property_info["address"]["location"].get("coordinate")
         )
+        list_date_str = property_info["basic"]["list_date"].split("T")[0] if property_info["basic"].get(
+            "list_date") else None
+        last_sold_date_str = property_info["basic"]["sold_date"].split("T")[0] if property_info["basic"].get(
+            "sold_date") else None
+
+        list_date = datetime.strptime(list_date_str, "%Y-%m-%d") if list_date_str else None
+        last_sold_date = datetime.strptime(last_sold_date_str, "%Y-%m-%d") if last_sold_date_str else None
+        today = datetime.now()
+
+        days_on_mls = None
+        status = property_info["basic"]["status"].lower()
+        if list_date:
+            if status == "sold" and last_sold_date:
+                days_on_mls = (last_sold_date - list_date).days
+            elif status in ('for_sale', 'for_rent'):
+                days_on_mls = (today - list_date).days
+            if days_on_mls and days_on_mls < 0:
+                days_on_mls = None
 
         listing = Property(
             mls=mls,
@@ -118,17 +143,13 @@ class RealtorScraper(Scraper):
             property_url=f"{self.PROPERTY_URL}{property_info['details']['permalink']}",
             status=property_info["basic"]["status"].upper(),
             list_price=property_info["basic"]["price"],
-            list_date=property_info["basic"]["list_date"].split("T")[0]
+            list_date=list_date,
-            if property_info["basic"].get("list_date")
-            else None,
             prc_sqft=property_info["basic"].get("price")
-            / property_info["basic"].get("sqft")
+                     / property_info["basic"].get("sqft")
             if property_info["basic"].get("price")
-            and property_info["basic"].get("sqft")
+               and property_info["basic"].get("sqft")
-            else None,
-            last_sold_date=property_info["basic"]["sold_date"].split("T")[0]
-            if property_info["basic"].get("sold_date")
             else None,
+            last_sold_date=last_sold_date,
             latitude=property_info["address"]["location"]["coordinate"].get("lat")
             if able_to_get_lat_long
             else None,
@@ -137,6 +158,8 @@ class RealtorScraper(Scraper):
             else None,
             address=self._parse_address(property_info, search_type="handle_listing"),
             description=Description(
+                primary_photo=property_info["primary_photo"].get("href", "").replace("s.jpg", "od-w480_h360_x2.webp?w=1080&q=75"),
+                alt_photos=self.process_alt_photos(property_info.get("photos", [])),
                 style=property_info["basic"].get("type", "").upper(),
                 beds=property_info["basic"].get("beds"),
                 baths_full=property_info["basic"].get("baths_full"),
@@ -148,6 +171,7 @@ class RealtorScraper(Scraper):
                 garage=property_info["details"].get("garage"),
                 stories=property_info["details"].get("stories"),
             ),
+            days_on_mls=days_on_mls
         )
 
         return [listing]
@@ -200,6 +224,7 @@ class RealtorScraper(Scraper):
                             stories
                         }
                         address {
+                            street_direction
                             street_number
                             street_name
                             street_suffix
@@ -230,6 +255,12 @@ class RealtorScraper(Scraper):
                             units
                             year_built
                         }
+                        primary_photo {
+                            href
+                        }
+                        photos {
+                            href
+                        }
                     }
                 }"""
 
@@ -300,6 +331,7 @@ class RealtorScraper(Scraper):
                                 }
                                 location {
                                     address {
+                                        street_direction
                                         street_number
                                         street_name
                                         street_suffix
@@ -316,19 +348,27 @@ class RealtorScraper(Scraper):
                                         name
                                     }
                                 }
+                                primary_photo {
+                                    href
+                                }
+                                photos {
+                                    href
+                                }
                             }
                         }
                     }"""
 
-        date_param = (
+        date_param = ""
-            'sold_date: { min: "$today-%sD" }' % self.last_x_days
+        if self.listing_type == ListingType.SOLD:
-            if self.listing_type == ListingType.SOLD and self.last_x_days
+            if self.date_from and self.date_to:
-            else (
+                date_param = f'sold_date: {{ min: "{self.date_from}", max: "{self.date_to}" }}'
-                'list_date: { min: "$today-%sD" }' % self.last_x_days
+            elif self.last_x_days:
-                if self.last_x_days
+                date_param = f'sold_date: {{ min: "$today-{self.last_x_days}D" }}'
-                else ""
+        else:
-            )
+            if self.date_from and self.date_to:
-        )
+                date_param = f'list_date: {{ min: "{self.date_from}", max: "{self.date_to}" }}'
+            elif self.last_x_days:
+                date_param = f'list_date: {{ min: "$today-{self.last_x_days}D" }}'
 
         sort_param = (
             "sort: [{ field: sold_date, direction: desc }]"
@@ -455,6 +495,9 @@ class RealtorScraper(Scraper):
 
             is_pending = result["flags"].get("is_pending") or result["flags"].get("is_contingent")
 
+            if is_pending and self.listing_type != ListingType.PENDING:
+                continue
+
             realty_property = Property(
                 mls=mls,
                 mls_id=result["source"].get("listing_id")
@@ -478,8 +521,8 @@ class RealtorScraper(Scraper):
                 if able_to_get_lat_long
                 else None,
                 address=self._parse_address(result, search_type="general_search"),
-                #: neighborhoods=self._parse_neighborhoods(result),
                 description=self._parse_description(result),
+                days_on_mls=self.calculate_days_on_mls(result)
             )
             properties.append(realty_property)
 
@@ -490,6 +533,9 @@ class RealtorScraper(Scraper):
 
     def search(self):
         location_info = self.handle_location()
+        if not location_info:
+            return []
+
         location_type = location_info["area_type"]
 
         search_variables = {
@@ -527,6 +573,11 @@ class RealtorScraper(Scraper):
                     "radius": "{}mi".format(self.radius),
                 }
 
+        elif location_type == "postal_code":
+            search_variables |= {
+                "postal_code": location_info.get("postal_code"),
+            }
+
         else:  #: general search, location
             search_variables |= {
                 "city": location_info.get("city"),
@@ -568,36 +619,53 @@ class RealtorScraper(Scraper):
         return ", ".join(neighborhoods_list) if neighborhoods_list else None
 
     @staticmethod
-    def _parse_address(result: dict, search_type):
+    def handle_none_safely(address_part):
+        if address_part is None:
+            return ""
+
+        return address_part
+
+    def _parse_address(self, result: dict, search_type):
         if search_type == "general_search":
-            return Address(
+            address = result['location']['address']
-                street=f"{result['location']['address']['street_number']} {result['location']['address']['street_name']} {result['location']['address']['street_suffix']}",
+        else:
-                unit=result["location"]["address"]["unit"],
+            address = result["address"]
-                city=result["location"]["address"]["city"],
+
-                state=result["location"]["address"]["state_code"],
-                zip=result["location"]["address"]["postal_code"],
-            )
         return Address(
-            street=f"{result['address']['street_number']} {result['address']['street_name']} {result['address']['street_suffix']}",
+            street=" ".join([
-            unit=result["address"]["unit"],
+                self.handle_none_safely(address.get('street_number')),
-            city=result["address"]["city"],
+                self.handle_none_safely(address.get('street_direction')),
-            state=result["address"]["state_code"],
+                self.handle_none_safely(address.get('street_name')),
-            zip=result["address"]["postal_code"],
+                self.handle_none_safely(address.get('street_suffix')),
+            ]).strip(),
+            unit=address["unit"],
+            city=address["city"],
+            state=address["state_code"],
+            zip=address["postal_code"],
         )
 
     @staticmethod
     def _parse_description(result: dict) -> Description:
+
         description_data = result.get("description", {})
 
         if description_data is None or not isinstance(description_data, dict):
-            print("Warning: description_data is invalid!")
             description_data = {}
 
         style = description_data.get("type", "")
         if style is not None:
             style = style.upper()
 
+        primary_photo = ""
+        if result and "primary_photo" in result:
+            primary_photo_info = result["primary_photo"]
+            if primary_photo_info and "href" in primary_photo_info:
+                primary_photo_href = primary_photo_info["href"]
+                primary_photo = primary_photo_href.replace("s.jpg", "od-w480_h360_x2.webp?w=1080&q=75")
+
         return Description(
+            primary_photo=primary_photo,
+            alt_photos=RealtorScraper.process_alt_photos(result.get("photos")),
             style=style,
             beds=description_data.get("beds"),
             baths_full=description_data.get("baths_full"),
@@ -609,3 +677,36 @@ class RealtorScraper(Scraper):
             garage=description_data.get("garage"),
             stories=description_data.get("stories"),
         )
+
+
+    @staticmethod
+    def calculate_days_on_mls(result: dict) -> Optional[int]:
+        list_date_str = result.get("list_date")
+        list_date = datetime.strptime(list_date_str.split("T")[0], "%Y-%m-%d") if list_date_str else None
+        last_sold_date_str = result.get("last_sold_date")
+        last_sold_date = datetime.strptime(last_sold_date_str, "%Y-%m-%d") if last_sold_date_str else None
+        today = datetime.now()
+
+        if list_date:
+            if result["status"] == 'sold':
+                if last_sold_date:
+                    days = (last_sold_date - list_date).days
+                    if days >= 0:
+                        return days
+            elif result["status"] in ('for_sale', 'for_rent'):
+                days = (today - list_date).days
+                if days >= 0:
+                    return days
+
+    @staticmethod
+    def process_alt_photos(photos_info):
+        try:
+            alt_photos = []
+            if photos_info:
+                for photo_info in photos_info:
+                    href = photo_info.get("href", "")
+                    alt_photo_href = href.replace("s.jpg", "od-w480_h360_x2.webp?w=1080&q=75")
+                    alt_photos.append(alt_photo_href)
+            return alt_photos
+        except Exception:
+            pass

homeharvest/exceptions.py

@@ -1,6 +1,5 @@
 class InvalidListingType(Exception):
     """Raised when a provided listing type is does not exist."""
 
-
+class InvalidDate(Exception):
-class NoResultsFound(Exception):
+    """Raised when only one of date_from or date_to is provided or not in the correct format. ex: 2023-10-23 """
-    """Raised when no results are found for the given location"""

homeharvest/utils.py

@@ -1,6 +1,7 @@
-from .core.scrapers.models import Property, ListingType
 import pandas as pd
-from .exceptions import InvalidListingType
+from datetime import datetime
+from .core.scrapers.models import Property, ListingType
+from .exceptions import InvalidListingType, InvalidDate
 
 ordered_properties = [
     "property_url",
@@ -18,6 +19,7 @@ ordered_properties = [
     "half_baths",
     "sqft",
     "year_built",
+    "days_on_mls",
     "list_price",
     "list_date",
     "sold_price",
@@ -29,6 +31,8 @@ ordered_properties = [
     "stories",
     "hoa_fee",
     "parking_garage",
+    "primary_photo",
+    "alt_photos",
 ]
 
 
@@ -47,6 +51,8 @@ def process_result(result: Property) -> pd.DataFrame:
     prop_data["price_per_sqft"] = prop_data["prc_sqft"]
 
     description = result.description
+    prop_data["primary_photo"] = description.primary_photo
+    prop_data["alt_photos"] = ", ".join(description.alt_photos)
     prop_data["style"] = description.style
     prop_data["beds"] = description.beds
     prop_data["full_baths"] = description.baths_full
@@ -69,3 +75,18 @@ def validate_input(listing_type: str) -> None:
         raise InvalidListingType(
             f"Provided listing type, '{listing_type}', does not exist."
         )
+
+
+def validate_dates(date_from: str | None, date_to: str | None) -> None:
+    if (date_from is not None and date_to is None) or (date_from is None and date_to is not None):
+        raise InvalidDate("Both date_from and date_to must be provided.")
+
+    if date_from and date_to:
+        try:
+            date_from_obj = datetime.strptime(date_from, "%Y-%m-%d")
+            date_to_obj = datetime.strptime(date_to, "%Y-%m-%d")
+
+            if date_to_obj < date_from_obj:
+                raise InvalidDate("date_to must be after date_from.")
+        except ValueError as e:
+            raise InvalidDate(f"Invalid date format or range")

pyproject.toml

@@ -1,9 +1,9 @@
 [tool.poetry]
 name = "homeharvest"
-version = "0.3.4"
+version = "0.3.11"
 description = "Real estate scraping library supporting Zillow, Realtor.com & Redfin."
 authors = ["Zachary Hampton <zachary@zacharysproducts.com>", "Cullen Watson <cullen@cullen.ai>"]
-homepage = "https://github.com/ZacharyHampton/HomeHarvest"
+homepage = "https://github.com/Bunsly/HomeHarvest"
 readme = "README.md"
 
 [tool.poetry.scripts]

tests/test_realtor.py

@@ -1,20 +1,15 @@
 from homeharvest import scrape_property
 from homeharvest.exceptions import (
     InvalidListingType,
-    NoResultsFound,
 )
 
 
 def test_realtor_pending_or_contingent():
     pending_or_contingent_result = scrape_property(
-        location="Surprise, AZ",
+        location="Surprise, AZ", listing_type="pending"
-        listing_type="pending"
     )
 
-    regular_result = scrape_property(
+    regular_result = scrape_property(location="Surprise, AZ", listing_type="for_sale")
-        location="Surprise, AZ",
-        listing_type="for_sale"
-    )
 
     assert all(
         [
@@ -54,6 +49,16 @@ def test_realtor_pending_comps():
     assert len(set([len(result) for result in results])) == len(results)
 
 
+def test_realtor_sold_past():
+    result = scrape_property(
+        location="San Diego, CA",
+        past_days=30,
+        listing_type="sold",
+    )
+
+    assert result is not None and len(result) > 0
+
+
 def test_realtor_comps():
     result = scrape_property(
         location="2530 Al Lipscomb Way",
@@ -79,6 +84,20 @@ def test_realtor_last_x_days_sold():
     ) and len(days_result_30) != len(days_result_10)
 
 
+def test_realtor_date_range_sold():
+    days_result_30 = scrape_property(
+        location="Dallas, TX", listing_type="sold", date_from="2023-05-01", date_to="2023-05-28"
+    )
+
+    days_result_60 = scrape_property(
+        location="Dallas, TX", listing_type="sold", date_from="2023-04-01", date_to="2023-06-10"
+    )
+
+    assert all(
+        [result is not None for result in [days_result_30, days_result_60]]
+    ) and len(days_result_30) < len(days_result_60)
+
+
 def test_realtor_single_property():
     results = [
         scrape_property(
@@ -111,15 +130,12 @@ def test_realtor():
 
     assert all([result is not None for result in results])
 
-    bad_results = []
+
-    try:
+def test_realtor_bad_address():
-        bad_results += [
+    bad_results = scrape_property(
-            scrape_property(
+            location="abceefg ju098ot498hh9",
-                location="abceefg ju098ot498hh9",
+            listing_type="for_sale",
-                listing_type="for_sale",
+        )
-            )
+    if len(bad_results) == 0:
-        ]
-    except (InvalidListingType, NoResultsFound):
         assert True
 
-    assert all([result is None for result in bad_results])