HomeHarvest is a simple, yet comprehensive, real estate scraping library that extracts and formats data in the style of MLS listings.

Not technical? Try out the web scraping tool on our site at tryhomeharvest.com.

Looking to build a data-focused software product? Book a call to work with us.

Check out another project we wrote: JobSpy – a Python package for job scraping

HomeHarvest Features

• Source: Fetches properties directly from Realtor.com.
• Data Format: Structures data to resemble MLS listings.
• Export Flexibility: Options to save as either CSV or Excel.
• Usage Modes:
  • CLI: For users who prefer command-line operations.
  • Python: For those who'd like to integrate scraping into their Python scripts.

Video Guide for HomeHarvest - updated for release v0.2.7

Installation

pip install homeharvest

Python version >= 3.10 required

Usage

CLI

usage: homeharvest [-l {for_sale,for_rent,sold}] [-o {excel,csv}] [-f FILENAME] [-p PROXY] [-d DAYS] [-r RADIUS] [-m] location
                                                                                                                             
Home Harvest Property Scraper                                                                                                 
                                                                                                                             
positional arguments:                                                                                                         
  location              Location to scrape (e.g., San Francisco, CA)                                                          
                                                                                                                             
options:                                                                                                                      
  -l {for_sale,for_rent,sold}, --listing_type {for_sale,for_rent,sold}                                                        
                        Listing type to scrape                                                                                
  -o {excel,csv}, --output {excel,csv}                                                                                        
                        Output format                                                                                         
  -f FILENAME, --filename FILENAME                                                                                            
                        Name of the output file (without extension)                                                           
  -p PROXY, --proxy PROXY                                                                                                     
                        Proxy to use for scraping                                                                             
  -d DAYS, --days DAYS  Sold in last _ days filter.                                                                           
  -r RADIUS, --radius RADIUS                                                                                                  
                        Get comparable properties within _ (eg. 0.0) miles. Only applicable for individual addresses.        
  -m, --mls_only        If set, fetches only MLS listings.

> homeharvest "San Francisco, CA" -l for_rent -o excel -f HomeHarvest

Python

from homeharvest import scrape_property
from datetime import datetime

# Generate filename based on current timestamp
current_timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"output/{current_timestamp}.csv"

properties = scrape_property(
    location="San Diego, CA",
    listing_type="sold", # for_sale, for_rent
    last_x_days=30, # sold/listed in last 30 days
    mls_only=True, # only fetch MLS listings
)
print(f"Number of properties: {len(properties)}")

# Export to csv
properties.to_csv(filename, index=False)
print(properties.head())

Output

>>> properties.head()
    MLS       MLS # Status          Style  ...     COEDate LotSFApx PrcSqft Stories
0  SDCA   230018348   SOLD         CONDOS  ...  2023-10-03   290110     803       2
1  SDCA   230016614   SOLD      TOWNHOMES  ...  2023-10-03     None     838       3
2  SDCA   230016367   SOLD         CONDOS  ...  2023-10-03    30056     649       1
3  MRCA  NDP2306335   SOLD  SINGLE_FAMILY  ...  2023-10-03     7519     661       2
4  SDCA   230014532   SOLD         CONDOS  ...  2023-10-03     None     752       1
[5 rows x 22 columns]

Parameters for scrape_property()

Required
├── location (str): The address in various formats - this could be just a zip code, a full address, or city/state, etc.
└── listing_type (option): Choose the type of listing.
    - 'for_rent'
    - 'for_sale'
    - 'sold'

Optional
├── radius (decimal): Radius in miles to find comparable properties based on individual addresses.
│    Example: 5.5 (fetches properties within a 5.5-mile radius if location is set to a specific address; otherwise, ignored)
│
├── last_x_days (integer): Number of past days to filter properties. Utilizes 'COEDate' for 'sold' listing types, and 'Lst Date' for others (for_rent, for_sale).
│    Example: 30 (fetches properties listed/sold in the last 30 days)
│
├── mls_only (True/False): If set, fetches only MLS listings (mainly applicable to 'sold' listings)
│
└── proxy (string): In format 'http://user:pass@host:port'

Property Schema

Property
├── Basic Information:
│ ├── property_url (str)
│ ├── mls (str)
│ ├── mls_id (str)
│ └── status (str)

├── Address Details:
│ ├── street
│ ├── unit
│ ├── city
│ ├── state
│ └── zip

├── Property Description:
│ ├── style
│ ├── beds
│ ├── baths_full
│ ├── baths_half
│ ├── sqft
│ ├── lot_sqft
│ ├── sold_price
│ ├── year_built
│ ├── garage
│ └── stories

├── Property Listing Details:
│ ├── list_price
│ ├── list_date
│ ├── last_sold_date
│ ├── prc_sqft
│ └── hoa_fee

├── Location Details:
│ ├── latitude
│ ├── longitude
│ └── neighborhoods

Exceptions

The following exceptions may be raised when using HomeHarvest:

• InvalidListingType - valid options: for_sale, for_rent, sold
• NoResultsFound - no properties found from your search

Frequently Asked Questions

---

Q: Encountering issues with your searches?
A: Try to broaden the parameters you're using. If problems persist, submit an issue.

---

Q: Received a Forbidden 403 response code?
A: This indicates that you have been blocked by Realtor.com for sending too many requests. We recommend:

• Waiting a few seconds between requests.
• Trying a VPN or useing a proxy as a parameter to scrape_property() to change your IP address.

---