fix(indeed): offset

chore:version
build(deps): bump markdownify to 0.13.1 (#211 )
2026-03-04 19:44:30 -08:00 · 2024-10-22 19:25:07 -05:00 · 2024-10-20 00:19:31 -05:00 · 2024-10-20 00:18:44 -05:00 · 2024-10-19 18:01:59 -05:00 · 2024-10-19 16:56:38 -05:00
31 changed files with 3870 additions and 2541 deletions
--- a/.github/workflows/python-test.yml
+++ b/.github/workflows/python-test.yml
@@ -0,0 +1,22 @@
+name: Python Tests
+
+on:
+  pull_request:
+    branches:
+      - main
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Set up Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: '3.8'
+      - name: Install dependencies
+        run: |
+          pip install poetry
+          poetry install
+      - name: Run tests
+        run: poetry run pytest tests/test_all.py
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -0,0 +1,7 @@
+repos:
+- repo: https://github.com/psf/black
+  rev: 24.2.0
+  hooks:
+  - id: black
+    language_version: python
+    args: [--line-length=88, --quiet]
--- a/README.md
+++ b/README.md
@@ -4,27 +4,21 @@

 **Not technical?** Try out the web scraping tool on our site at [usejobspy.com](https://usejobspy.com).

-*Looking to build a data-focused software product?* **[Book a call](https://calendly.com/zachary-products/15min)** *to
+*Looking to build a data-focused software product?* **[Book a call](https://bunsly.com/)** *to
 work with us.*
-\
-Check out another project we wrote: ***[HomeHarvest](https://github.com/ZacharyHampton/HomeHarvest)** – a Python package
-for real estate scraping*

 ## Features

- Scrapes job postings from **LinkedIn**, **Indeed** & **ZipRecruiter** simultaneously
+- Scrapes job postings from **LinkedIn**, **Indeed**, **Glassdoor**, & **ZipRecruiter** simultaneously
 - Aggregates the job postings in a Pandas DataFrame
- Proxy support (HTTP/S, SOCKS)
-
-[Video Guide for JobSpy](https://www.youtube.com/watch?v=RuP1HrAZnxs&pp=ygUgam9icyBzY3JhcGVyIGJvdCBsaW5rZWRpbiBpbmRlZWQ%3D) -
-Updated for release v1.1.3
+- Proxies support

 ![jobspy](https://github.com/cullenwatson/JobSpy/assets/78247585/ec7ef355-05f6-4fd3-8161-a817e31c5c57)

 ### Installation

 ```
-pip install --upgrade python-jobspy
+pip install -U python-jobspy
 ```

 _Python version >= [3.10](https://www.python.org/downloads/release/python-3100/) required_
@@ -32,28 +26,30 @@ _Python version >= [3.10](https://www.python.org/downloads/release/python-3100/)
 ### Usage

 ```python
+import csv
 from jobspy import scrape_jobs

 jobs = scrape_jobs(
-    site_name=["indeed", "linkedin", "zip_recruiter"],
+    site_name=["indeed", "linkedin", "zip_recruiter", "glassdoor"],
    search_term="software engineer",
    location="Dallas, TX",
-    results_wanted=10,
-    country_indeed='USA'  # only needed for indeed
+    results_wanted=20,
+    hours_old=72, # (only Linkedin/Indeed is hour specific, others round up to days old)
+    country_indeed='USA',  # only needed for indeed / glassdoor
+    
+    # linkedin_fetch_description=True # get more info such as full description, direct job url for linkedin (slower)
+    # proxies=["208.195.175.46:65095", "208.195.175.45:65095", "localhost"],
+    
 )
 print(f"Found {len(jobs)} jobs")
 print(jobs.head())
-jobs.to_csv("jobs.csv", index=False)
-
-# output to .xlsx
-# jobs.to_xlsx('jobs.xlsx', index=False)
-
+jobs.to_csv("jobs.csv", quoting=csv.QUOTE_NONNUMERIC, escapechar="\\", index=False) # to_excel
 ```

 ### Output

 ```
-SITE           TITLE                             COMPANY_NAME      CITY          STATE  JOB_TYPE  INTERVAL  MIN_AMOUNT  MAX_AMOUNT  JOB_URL                                            DESCRIPTION
+SITE           TITLE                             COMPANY           CITY          STATE  JOB_TYPE  INTERVAL  MIN_AMOUNT  MAX_AMOUNT  JOB_URL                                            DESCRIPTION
 indeed         Software Engineer                 AMERICAN SYSTEMS  Arlington     VA     None      yearly    200000      150000      https://www.indeed.com/viewjob?jk=5e409e577046...  THIS POSITION COMES WITH A 10K SIGNING BONUS!...
 indeed         Senior Software Engineer          TherapyNotes.com  Philadelphia  PA     fulltime  yearly    135000      110000      https://www.indeed.com/viewjob?jk=da39574a40cb...  About Us TherapyNotes is the national leader i...
 linkedin       Software Engineer - Early Career  Lockheed Martin   Sunnyvale     CA     fulltime  yearly    None        None        https://www.linkedin.com/jobs/view/3693012711      Description:By bringing together people that u...
@@ -65,53 +61,116 @@ zip_recruiter Software Developer                 TEKsystems        Phoenix
 ### Parameters for `scrape_jobs()`

 ```plaintext
-Required
-├── site_type (List[enum]): linkedin, zip_recruiter, indeed
-└── search_term (str)
 Optional
-├── location (int)
-├── distance (int): in miles
-├── job_type (enum): fulltime, parttime, internship, contract
-├── proxy (str): in format 'http://user:pass@host:port' or [https, socks]
+├── site_name (list|str): 
+|    linkedin, zip_recruiter, indeed, glassdoor 
+|    (default is all four)
+│
+├── search_term (str)
+│
+├── location (str)
+│
+├── distance (int): 
+|    in miles, default 50
+│
+├── job_type (str): 
+|    fulltime, parttime, internship, contract
+│
+├── proxies (list): 
+|    in format ['user:pass@host:port', 'localhost']
+|    each job board scraper will round robin through the proxies
+|
+├── ca_cert (str)
+|    path to CA Certificate file for proxies
+│
 ├── is_remote (bool)
-├── results_wanted (int): number of job results to retrieve for each site specified in 'site_type'
-├── easy_apply (bool): filters for jobs that are hosted on LinkedIn
-├── country_indeed (enum): filters the country on Indeed (see below for correct spelling)
-├── offset (num): starts the search from an offset (e.g. 25 will start the search from the 25th result)
+│
+├── results_wanted (int): 
+|    number of job results to retrieve for each site specified in 'site_name'
+│
+├── easy_apply (bool): 
+|    filters for jobs that are hosted on the job board site
+│
+├── description_format (str): 
+|    markdown, html (Format type of the job descriptions. Default is markdown.)
+│
+├── offset (int): 
+|    starts the search from an offset (e.g. 25 will start the search from the 25th result)
+│
+├── hours_old (int): 
+|    filters jobs by the number of hours since the job was posted 
+|    (ZipRecruiter and Glassdoor round up to next day.)
+│
+├── verbose (int) {0, 1, 2}: 
+|    Controls the verbosity of the runtime printouts 
+|    (0 prints only errors, 1 is errors+warnings, 2 is all logs. Default is 2.)
+
+├── linkedin_fetch_description (bool): 
+|    fetches full description and direct job url for LinkedIn (Increases requests by O(n))
+│
+├── linkedin_company_ids (list[int]): 
+|    searches for linkedin jobs with specific company ids
+|
+├── country_indeed (str): 
+|    filters the country on Indeed & Glassdoor (see below for correct spelling)
+|
+├── enforce_annual_salary (bool): 
+|    converts wages to annual salary
 ```

+```
+├── Indeed limitations:
+|    Only one from this list can be used in a search:
+|    - hours_old
+|    - job_type & is_remote
+|    - easy_apply
+│
+└── LinkedIn limitations:
+|    Only one from this list can be used in a search:
+|    - hours_old
+|    - easy_apply
+```
+
+
 ### JobPost Schema

 ```plaintext
 JobPost
-├── title (str)
-├── company (str)
-├── job_url (str)
-├── location (object)
-│   ├── country (str)
-│   ├── city (str)
-│   ├── state (str)
-├── description (str)
-├── job_type (enum): fulltime, parttime, internship, contract
-├── compensation (object)
-│   ├── interval (enum): yearly, monthly, weekly, daily, hourly
-│   ├── min_amount (int)
-│   ├── max_amount (int)
-│   └── currency (enum)
-└── date_posted (date)
-└── emails (str)
-└── num_urgent_words (int)
-└── is_remote (bool) - just for Indeed at the momen
+├── title
+├── company
+├── company_url
+├── job_url
+├── location
+│   ├── country
+│   ├── city
+│   ├── state
+├── description
+├── job_type: fulltime, parttime, internship, contract
+├── job_function
+│   ├── interval: yearly, monthly, weekly, daily, hourly
+│   ├── min_amount
+│   ├── max_amount
+│   ├── currency
+│   └── salary_source: direct_data, description (parsed from posting)
+├── date_posted
+├── emails
+└── is_remote
+
+Linkedin specific
+└── job_level
+
+Linkedin & Indeed specific
+└── company_industry
+
+Indeed specific
+├── company_country
+├── company_addresses
+├── company_employees_label
+├── company_revenue_label
+├── company_description
+└── logo_photo_url
 ```

-### Exceptions
-
-The following exceptions may be raised when using JobSpy:
-
-* `LinkedInException`
-* `IndeedException`
-* `ZipRecruiterException`
-
 ## Supported Countries for Job Searching

 ### **LinkedIn**
@@ -122,56 +181,56 @@ LinkedIn searches globally & uses only the `location` parameter.

 ZipRecruiter searches for jobs in **US/Canada** & uses only the `location` parameter.

-### **Indeed**
+### **Indeed / Glassdoor**

-Indeed supports most countries, but the `country_indeed` parameter is required. Additionally, use the `location`
+Indeed & Glassdoor supports most countries, but the `country_indeed` parameter is required. Additionally, use the `location`
 parameter to narrow down the location, e.g. city & state if necessary. 

-You can specify the following countries when searching on Indeed (use the exact name):
+You can specify the following countries when searching on Indeed (use the exact name, * indicates support for Glassdoor):

 |                      |              |            |                |
 |----------------------|--------------|------------|----------------|
-| Argentina            | Australia    | Austria    | Bahrain        |
-| Belgium              | Brazil       | Canada     | Chile          |
+| Argentina            | Australia*   | Austria*   | Bahrain        |
+| Belgium*             | Brazil*      | Canada*    | Chile          |
 | China                | Colombia     | Costa Rica | Czech Republic |
 | Denmark              | Ecuador      | Egypt      | Finland        |
-| France               | Germany      | Greece     | Hong Kong      |
-| Hungary              | India        | Indonesia  | Ireland        |
-| Israel               | Italy        | Japan      | Kuwait         |
-| Luxembourg           | Malaysia     | Mexico     | Morocco        |
-| Netherlands          | New Zealand  | Nigeria    | Norway         |
+| France*              | Germany*     | Greece     | Hong Kong*     |
+| Hungary              | India*       | Indonesia  | Ireland*       |
+| Israel               | Italy*       | Japan      | Kuwait         |
+| Luxembourg           | Malaysia     | Mexico*    | Morocco        |
+| Netherlands*         | New Zealand* | Nigeria    | Norway         |
 | Oman                 | Pakistan     | Panama     | Peru           |
 | Philippines          | Poland       | Portugal   | Qatar          |
-| Romania              | Saudi Arabia | Singapore  | South Africa   |
-| South Korea          | Spain        | Sweden     | Switzerland    |
+| Romania              | Saudi Arabia | Singapore* | South Africa   |
+| South Korea          | Spain*       | Sweden     | Switzerland*   |
 | Taiwan               | Thailand     | Turkey     | Ukraine        |
-| United Arab Emirates | UK           | USA        | Uruguay        |
-| Venezuela            | Vietnam      |            |                |
+| United Arab Emirates | UK*          | USA*       | Uruguay        |
+| Venezuela            | Vietnam*     |            |                |
+
+
+## Notes
+* Indeed is the best scraper currently with no rate limiting.  
+* All the job board endpoints are capped at around 1000 jobs on a given search.  
+* LinkedIn is the most restrictive and usually rate limits around the 10th page with one ip. Proxies are a must basically.

 ## Frequently Asked Questions

+---
+**Q: Why is Indeed giving unrelated roles?**  
+**A:** Indeed is searching each one of your terms e.g. software intern, it searches software OR intern. Try search_term='"software intern"' in quotes for stricter searching
+
+---
+
+**Q: Received a response code 429?**  
+**A:** This indicates that you have been blocked by the job board site for sending too many requests. All of the job board sites are aggressive with blocking. We recommend:
+
+- Wait some time between scrapes (site-dependent).
+- Try using the proxies param to change your IP address.
+
 ---

 **Q: Encountering issues with your queries?**  
 **A:** Try reducing the number of `results_wanted` and/or broadening the filters. If problems
-persist, [submit an issue](https://github.com/cullenwatson/JobSpy/issues).
+persist, [submit an issue](https://github.com/Bunsly/JobSpy/issues).

 ---
-
-**Q: Received a response code 429?**  
-**A:** This indicates that you have been blocked by the job board site for sending too many requests. Currently, *
-*LinkedIn** is particularly aggressive with blocking. We recommend:
-
- Waiting a few seconds between requests.
- Trying a VPN or proxy to change your IP address.
-
---
-
-**Q: Experiencing a "Segmentation fault: 11" on macOS Catalina?**  
-**A:** This is due to `tls_client` dependency not supporting your architecture. Solutions and workarounds include:
-
- Upgrade to a newer version of MacOS
- Reach out to the maintainers of [tls_client](https://github.com/bogdanfinn/tls-client) for fixes
-
-
-  
--- a/examples/JobSpy_Demo.ipynb
+++ b/examples/JobSpy_Demo.ipynb
@@ -1,167 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "00a94b47-f47b-420f-ba7e-714ef219c006",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "from jobspy import scrape_jobs\n",
-    "import pandas as pd\n",
-    "from IPython.display import display, HTML"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "9f773e6c-d9fc-42cc-b0ef-63b739e78435",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "pd.set_option('display.max_columns', None)\n",
-    "pd.set_option('display.max_rows', None)\n",
-    "pd.set_option('display.width', None)\n",
-    "pd.set_option('display.max_colwidth', 50)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "1253c1f8-9437-492e-9dd3-e7fe51099420",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# example 1 (no hyperlinks, USA)\n",
-    "jobs = scrape_jobs(\n",
-    "    site_name=[\"linkedin\"],\n",
-    "    location='san francisco',\n",
-    "    search_term=\"engineer\",\n",
-    "    results_wanted=5,\n",
-    "\n",
-    "    # use if you want to use a proxy\n",
-    "    # proxy=\"socks5://jobspy:5a4vpWtj4EeJ2hoYzk@us.smartproxy.com:10001\",\n",
-    "    proxy=\"http://jobspy:5a4vpWtj4EeJ2hoYzk@us.smartproxy.com:10001\",\n",
-    "    #proxy=\"https://jobspy:5a4vpWtj4EeJ2hoYzk@us.smartproxy.com:10001\",\n",
-    ")\n",
-    "display(jobs)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "6a581b2d-f7da-4fac-868d-9efe143ee20a",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# example 2 - remote USA & hyperlinks\n",
-    "jobs = scrape_jobs(\n",
-    "    site_name=[\"linkedin\", \"zip_recruiter\", \"indeed\"],\n",
-    "    # location='san francisco',\n",
-    "    search_term=\"software engineer\",\n",
-    "    country_indeed=\"USA\",\n",
-    "    hyperlinks=True,\n",
-    "    is_remote=True,\n",
-    "    results_wanted=5, \n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "fe8289bc-5b64-4202-9a64-7c117c83fd9a",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# use if hyperlinks=True\n",
-    "html = jobs.to_html(escape=False)\n",
-    "# change max-width: 200px to show more or less of the content\n",
-    "truncate_width = f'<style>.dataframe td {{ max-width: 200px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }}</style>{html}'\n",
-    "display(HTML(truncate_width))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "951c2fe1-52ff-407d-8bb1-068049b36777",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# example 3 - with hyperlinks, international - linkedin (no zip_recruiter)\n",
-    "jobs = scrape_jobs(\n",
-    "    site_name=[\"linkedin\"],\n",
-    "    location='berlin',\n",
-    "    search_term=\"engineer\",\n",
-    "    hyperlinks=True,\n",
-    "    results_wanted=5,\n",
-    "    easy_apply=True\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "1e37a521-caef-441c-8fc2-2eb5b2e7da62",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# use if hyperlinks=True\n",
-    "html = jobs.to_html(escape=False)\n",
-    "# change max-width: 200px to show more or less of the content\n",
-    "truncate_width = f'<style>.dataframe td {{ max-width: 200px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }}</style>{html}'\n",
-    "display(HTML(truncate_width))"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "0650e608-0b58-4bf5-ae86-68348035b16a",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# example 4 - international indeed (no zip_recruiter)\n",
-    "jobs = scrape_jobs(\n",
-    "    site_name=[\"indeed\"],\n",
-    "    search_term=\"engineer\",\n",
-    "    country_indeed = \"China\",\n",
-    "    hyperlinks=True\n",
-    ")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "40913ac8-3f8a-4d7e-ac47-afb88316432b",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "# use if hyperlinks=True\n",
-    "html = jobs.to_html(escape=False)\n",
-    "# change max-width: 200px to show more or less of the content\n",
-    "truncate_width = f'<style>.dataframe td {{ max-width: 200px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }}</style>{html}'\n",
-    "display(HTML(truncate_width))"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
-   "language": "python",
-   "name": "python3"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.11.5"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
--- a/examples/JobSpy_Demo.py
+++ b/examples/JobSpy_Demo.py
@@ -1,31 +0,0 @@
-from jobspy import scrape_jobs
-import pandas as pd
-
-jobs: pd.DataFrame = scrape_jobs(
-    site_name=["indeed", "linkedin", "zip_recruiter"],
-    search_term="software engineer",
-    location="Dallas, TX",
-    results_wanted=50,  # be wary the higher it is, the more likey you'll get blocked (rotating proxy should work tho)
-    country_indeed="USA",
-    offset=25  # start jobs from an offset (use if search failed and want to continue)
-    # proxy="http://jobspy:5a4vpWtj8EeJ2hoYzk@ca.smartproxy.com:20001",
-)
-
-# formatting for pandas
-pd.set_option("display.max_columns", None)
-pd.set_option("display.max_rows", None)
-pd.set_option("display.width", None)
-pd.set_option("display.max_colwidth", 50)  # set to 0 to see full job url / desc
-
-# 1: output to console
-print(jobs)
-
-# 2: output to .csv
-jobs.to_csv("./jobs.csv", index=False)
-print("outputted to jobs.csv")
-
-# 3: output to .xlsx
-# jobs.to_xlsx('jobs.xlsx', index=False)
-
-# 4: display in Jupyter Notebook (1. pip install jupyter 2. jupyter notebook)
-# display(jobs)
--- a/poetry.lock
+++ b/poetry.lock
--- a/poetry.toml
+++ b/poetry.toml
@@ -0,0 +1,2 @@
+[virtualenvs]
+in-project = true
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,10 +1,11 @@
 [tool.poetry]
 name = "python-jobspy"
-version = "1.1.13"
-description = "Job scraper for LinkedIn, Indeed & ZipRecruiter"
-authors = ["Zachary Hampton <zachary@zacharysproducts.com>", "Cullen Watson <cullen@cullen.ai>"]
-homepage = "https://github.com/cullenwatson/JobSpy"
+version = "1.1.72"
+description = "Job scraper for LinkedIn, Indeed, Glassdoor & ZipRecruiter"
+authors = ["Zachary Hampton <zachary@bunsly.com>", "Cullen Watson <cullen@bunsly.com>"]
+homepage = "https://github.com/Bunsly/JobSpy"
 readme = "README.md"
+keywords = ['jobs-scraper', 'linkedin', 'indeed', 'glassdoor', 'ziprecruiter']

 packages = [
    { include = "jobspy", from = "src" }
@@ -13,16 +14,24 @@ packages = [
 [tool.poetry.dependencies]
 python = "^3.10"
 requests = "^2.31.0"
-tls-client = "^0.2.1"
 beautifulsoup4 = "^4.12.2"
 pandas = "^2.1.0"
+NUMPY = "1.26.3"
 pydantic = "^2.3.0"
+tls-client = "^1.0.1"
+markdownify = "^0.13.1"
+regex = "^2024.4.28"


 [tool.poetry.group.dev.dependencies]
 pytest = "^7.4.1"
 jupyter = "^1.0.0"
+black = "*"
+pre-commit = "*"

 [build-system]
 requires = ["poetry-core"]
 build-backend = "poetry.core.masonry.api"
+
+[tool.black]
+line-length = 88
--- a/src/jobspy/init.py
+++ b/src/jobspy/init.py
@@ -1,49 +1,61 @@
+from __future__ import annotations
+
 import pandas as pd
-import concurrent.futures
-from concurrent.futures import ThreadPoolExecutor
-from typing import Tuple, Optional
+from typing import Tuple
+from concurrent.futures import ThreadPoolExecutor, as_completed

 from .jobs import JobType, Location
+from .scrapers.utils import set_logger_level, extract_salary, create_logger
 from .scrapers.indeed import IndeedScraper
 from .scrapers.ziprecruiter import ZipRecruiterScraper
+from .scrapers.glassdoor import GlassdoorScraper
 from .scrapers.linkedin import LinkedInScraper
-from .scrapers import ScraperInput, Site, JobResponse, Country
+from .scrapers import SalarySource, ScraperInput, Site, JobResponse, Country
 from .scrapers.exceptions import (
    LinkedInException,
    IndeedException,
    ZipRecruiterException,
+    GlassdoorException,
 )

+
+def scrape_jobs(
+    site_name: str | list[str] | Site | list[Site] | None = None,
+    search_term: str | None = None,
+    location: str | None = None,
+    distance: int | None = 50,
+    is_remote: bool = False,
+    job_type: str | None = None,
+    easy_apply: bool | None = None,
+    results_wanted: int = 15,
+    country_indeed: str = "usa",
+    hyperlinks: bool = False,
+    proxies: list[str] | str | None = None,
+    ca_cert: str | None = None,
+    description_format: str = "markdown",
+    linkedin_fetch_description: bool | None = False,
+    linkedin_company_ids: list[int] | None = None,
+    offset: int | None = 0,
+    hours_old: int = None,
+    enforce_annual_salary: bool = False,
+    verbose: int = 2,
+    **kwargs,
+) -> pd.DataFrame:
+    """
+    Simultaneously scrapes job data from multiple job sites.
+    :return: pandas dataframe containing job data
+    """
    SCRAPER_MAPPING = {
        Site.LINKEDIN: LinkedInScraper,
        Site.INDEED: IndeedScraper,
        Site.ZIP_RECRUITER: ZipRecruiterScraper,
+        Site.GLASSDOOR: GlassdoorScraper,
    }
+    set_logger_level(verbose)

-
-def _map_str_to_site(site_name: str) -> Site:
+    def map_str_to_site(site_name: str) -> Site:
        return Site[site_name.upper()]

-
-def scrape_jobs(
-    site_name: str | list[str] | Site | list[Site],
-    search_term: str,
-    location: str = "",
-    distance: int = None,
-    is_remote: bool = False,
-    job_type: str = None,
-    easy_apply: bool = False,  # linkedin
-    results_wanted: int = 15,
-    country_indeed: str = "usa",
-    hyperlinks: bool = False,
-    proxy: Optional[str] = None,
-    offset: Optional[int] = 0,
-) -> pd.DataFrame:
-    """
-    Simultaneously scrapes job data from multiple job sites.
-    :return: results_wanted: pandas dataframe containing job data
-    """
-
    def get_enum_from_value(value_str):
        for job_type in JobType:
            if value_str in job_type.value:
@@ -52,18 +64,23 @@ def scrape_jobs(

    job_type = get_enum_from_value(job_type) if job_type else None

-    if type(site_name) == str:
-        site_type = [_map_str_to_site(site_name)]
-    else:  #: if type(site_name) == list
-        site_type = [
-            _map_str_to_site(site) if type(site) == str else site_name
+    def get_site_type():
+        site_types = list(Site)
+        if isinstance(site_name, str):
+            site_types = [map_str_to_site(site_name)]
+        elif isinstance(site_name, Site):
+            site_types = [site_name]
+        elif isinstance(site_name, list):
+            site_types = [
+                map_str_to_site(site) if isinstance(site, str) else site
                for site in site_name
            ]
+        return site_types

    country_enum = Country.from_string(country_indeed)

    scraper_input = ScraperInput(
-        site_type=site_type,
+        site_type=get_site_type(),
        country=country_enum,
        search_term=search_term,
        location=location,
@@ -71,28 +88,21 @@ def scrape_jobs(
        is_remote=is_remote,
        job_type=job_type,
        easy_apply=easy_apply,
+        description_format=description_format,
+        linkedin_fetch_description=linkedin_fetch_description,
        results_wanted=results_wanted,
+        linkedin_company_ids=linkedin_company_ids,
        offset=offset,
+        hours_old=hours_old,
    )

    def scrape_site(site: Site) -> Tuple[str, JobResponse]:
        scraper_class = SCRAPER_MAPPING[site]
-        scraper = scraper_class(proxy=proxy)
-
-        try:
+        scraper = scraper_class(proxies=proxies, ca_cert=ca_cert)
        scraped_data: JobResponse = scraper.scrape(scraper_input)
-        except (LinkedInException, IndeedException, ZipRecruiterException) as lie:
-            raise lie
-        except Exception as e:
-            # unhandled exceptions
-            if site == Site.LINKEDIN:
-                raise LinkedInException()
-            if site == Site.INDEED:
-                raise IndeedException()
-            if site == Site.ZIP_RECRUITER:
-                raise ZipRecruiterException()
-            else:
-                raise e
+        cap_name = site.value.capitalize()
+        site_name = "ZipRecruiter" if cap_name == "Zip_recruiter" else cap_name
+        create_logger(site_name).info(f"finished scraping")
        return site.value, scraped_data

    site_to_jobs_dict = {}
@@ -106,18 +116,32 @@ def scrape_jobs(
            executor.submit(worker, site): site for site in scraper_input.site_type
        }

-        for future in concurrent.futures.as_completed(future_to_site):
+        for future in as_completed(future_to_site):
            site_value, scraped_data = future.result()
            site_to_jobs_dict[site_value] = scraped_data

+    def convert_to_annual(job_data: dict):
+        if job_data["interval"] == "hourly":
+            job_data["min_amount"] *= 2080
+            job_data["max_amount"] *= 2080
+        if job_data["interval"] == "monthly":
+            job_data["min_amount"] *= 12
+            job_data["max_amount"] *= 12
+        if job_data["interval"] == "weekly":
+            job_data["min_amount"] *= 52
+            job_data["max_amount"] *= 52
+        if job_data["interval"] == "daily":
+            job_data["min_amount"] *= 260
+            job_data["max_amount"] *= 260
+        job_data["interval"] = "yearly"
+
    jobs_dfs: list[pd.DataFrame] = []

    for site, job_response in site_to_jobs_dict.items():
        for job in job_response.jobs:
            job_data = job.dict()
-            job_data[
-                "job_url_hyper"
-            ] = f'<a href="{job_data["job_url"]}">{job_data["job_url"]}</a>'
+            job_url = job_data["job_url"]
+            job_data["job_url_hyper"] = f'<a href="{job_url}">{job_url}</a>'
            job_data["site"] = site
            job_data["company"] = job_data["company_name"]
            job_data["job_type"] = (
@@ -128,7 +152,10 @@ def scrape_jobs(
            job_data["emails"] = (
                ", ".join(job_data["emails"]) if job_data["emails"] else None
            )
-            job_data["location"] = Location(**job_data["location"]).display_location()
+            if job_data["location"]:
+                job_data["location"] = Location(
+                    **job_data["location"]
+                ).display_location()

            compensation_obj = job_data.get("compensation")
            if compensation_obj and isinstance(compensation_obj, dict):
@@ -140,37 +167,84 @@ def scrape_jobs(
                job_data["min_amount"] = compensation_obj.get("min_amount")
                job_data["max_amount"] = compensation_obj.get("max_amount")
                job_data["currency"] = compensation_obj.get("currency", "USD")
-            else:
-                job_data["interval"] = None
-                job_data["min_amount"] = None
-                job_data["max_amount"] = None
-                job_data["currency"] = None
+                job_data["salary_source"] = SalarySource.DIRECT_DATA.value
+                if enforce_annual_salary and (
+                    job_data["interval"]
+                    and job_data["interval"] != "yearly"
+                    and job_data["min_amount"]
+                    and job_data["max_amount"]
+                ):
+                    convert_to_annual(job_data)

+            else:
+                if country_enum == Country.USA:
+                    (
+                        job_data["interval"],
+                        job_data["min_amount"],
+                        job_data["max_amount"],
+                        job_data["currency"],
+                    ) = extract_salary(
+                        job_data["description"],
+                        enforce_annual_salary=enforce_annual_salary,
+                    )
+                    job_data["salary_source"] = SalarySource.DESCRIPTION.value
+
+            job_data["salary_source"] = (
+                job_data["salary_source"]
+                if "min_amount" in job_data and job_data["min_amount"]
+                else None
+            )
            job_df = pd.DataFrame([job_data])
            jobs_dfs.append(job_df)

    if jobs_dfs:
-        jobs_df = pd.concat(jobs_dfs, ignore_index=True)
-        desired_order: list[str] = [
-            "job_url_hyper" if hyperlinks else "job_url",
+        # Step 1: Filter out all-NA columns from each DataFrame before concatenation
+        filtered_dfs = [df.dropna(axis=1, how="all") for df in jobs_dfs]
+
+        # Step 2: Concatenate the filtered DataFrames
+        jobs_df = pd.concat(filtered_dfs, ignore_index=True)
+
+        # Desired column order
+        desired_order = [
+            "id",
            "site",
+            "job_url_hyper" if hyperlinks else "job_url",
+            "job_url_direct",
            "title",
            "company",
            "location",
            "job_type",
            "date_posted",
+            "salary_source",
            "interval",
            "min_amount",
            "max_amount",
            "currency",
            "is_remote",
-            "num_urgent_words",
-            "benefits",
+            "job_level",
+            "job_function",
+            "company_industry",
+            "listing_type",
            "emails",
            "description",
+            "company_url",
+            "logo_photo_url",
+            "company_url_direct",
+            "company_addresses",
+            "company_num_employees",
+            "company_revenue",
+            "company_description",
        ]
-        jobs_formatted_df = jobs_df[desired_order]
-    else:
-        jobs_formatted_df = pd.DataFrame()

-    return jobs_formatted_df
+        # Step 3: Ensure all desired columns are present, adding missing ones as empty
+        for column in desired_order:
+            if column not in jobs_df.columns:
+                jobs_df[column] = None  # Add missing columns as empty
+
+        # Reorder the DataFrame according to the desired order
+        jobs_df = jobs_df[desired_order]
+
+        # Step 4: Sort the DataFrame as required
+        return jobs_df.sort_values(by=["site", "date_posted"], ascending=[True, False])
+    else:
+        return pd.DataFrame()
--- a/src/jobspy/jobs/init.py
+++ b/src/jobspy/jobs/init.py
@@ -1,8 +1,9 @@
-from typing import Union, Optional
+from __future__ import annotations
+
+from typing import Optional
 from datetime import date
 from enum import Enum
-
-from pydantic import BaseModel, validator
+from pydantic import BaseModel


 class JobType(Enum):
@@ -37,10 +38,16 @@ class JobType(Enum):
        "повназайнятість",
        "toànthờigian",
    )
-    PART_TIME = ("parttime", "teilzeit")
+    PART_TIME = ("parttime", "teilzeit", "částečnýúvazek", "deltid")
    CONTRACT = ("contract", "contractor")
    TEMPORARY = ("temporary",)
-    INTERNSHIP = ("internship", "prácticas", "ojt(onthejobtraining)", "praktikum")
+    INTERNSHIP = (
+        "internship",
+        "prácticas",
+        "ojt(onthejobtraining)",
+        "praktikum",
+        "praktik",
+    )

    PER_DIEM = ("perdiem",)
    NIGHTS = ("nights",)
@@ -50,40 +57,47 @@ class JobType(Enum):


 class Country(Enum):
-    ARGENTINA = ("argentina", "ar")
-    AUSTRALIA = ("australia", "au")
-    AUSTRIA = ("austria", "at")
+    """
+    Gets the subdomain for Indeed and Glassdoor.
+    The second item in the tuple is the subdomain (and API country code if there's a ':' separator) for Indeed
+    The third item in the tuple is the subdomain (and tld if there's a ':' separator) for Glassdoor
+    """
+
+    ARGENTINA = ("argentina", "ar", "com.ar")
+    AUSTRALIA = ("australia", "au", "com.au")
+    AUSTRIA = ("austria", "at", "at")
    BAHRAIN = ("bahrain", "bh")
-    BELGIUM = ("belgium", "be")
-    BRAZIL = ("brazil", "br")
-    CANADA = ("canada", "ca")
+    BELGIUM = ("belgium", "be", "fr:be")
+    BRAZIL = ("brazil", "br", "com.br")
+    CANADA = ("canada", "ca", "ca")
    CHILE = ("chile", "cl")
    CHINA = ("china", "cn")
    COLOMBIA = ("colombia", "co")
    COSTARICA = ("costa rica", "cr")
-    CZECHREPUBLIC = ("czech republic", "cz")
+    CZECHREPUBLIC = ("czech republic,czechia", "cz")
    DENMARK = ("denmark", "dk")
    ECUADOR = ("ecuador", "ec")
    EGYPT = ("egypt", "eg")
    FINLAND = ("finland", "fi")
-    FRANCE = ("france", "fr")
-    GERMANY = ("germany", "de")
+    FRANCE = ("france", "fr", "fr")
+    GERMANY = ("germany", "de", "de")
    GREECE = ("greece", "gr")
-    HONGKONG = ("hong kong", "hk")
+    HONGKONG = ("hong kong", "hk", "com.hk")
    HUNGARY = ("hungary", "hu")
-    INDIA = ("india", "in")
+    INDIA = ("india", "in", "co.in")
    INDONESIA = ("indonesia", "id")
-    IRELAND = ("ireland", "ie")
+    IRELAND = ("ireland", "ie", "ie")
    ISRAEL = ("israel", "il")
-    ITALY = ("italy", "it")
+    ITALY = ("italy", "it", "it")
    JAPAN = ("japan", "jp")
    KUWAIT = ("kuwait", "kw")
    LUXEMBOURG = ("luxembourg", "lu")
-    MALAYSIA = ("malaysia", "malaysia")
-    MEXICO = ("mexico", "mx")
+    MALAYSIA = ("malaysia", "malaysia:my", "com")
+    MALTA = ("malta", "malta:mt", "mt")
+    MEXICO = ("mexico", "mx", "com.mx")
    MOROCCO = ("morocco", "ma")
-    NETHERLANDS = ("netherlands", "nl")
-    NEWZEALAND = ("new zealand", "nz")
+    NETHERLANDS = ("netherlands", "nl", "nl")
+    NEWZEALAND = ("new zealand", "nz", "co.nz")
    NIGERIA = ("nigeria", "ng")
    NORWAY = ("norway", "no")
    OMAN = ("oman", "om")
@@ -96,54 +110,66 @@ class Country(Enum):
    QATAR = ("qatar", "qa")
    ROMANIA = ("romania", "ro")
    SAUDIARABIA = ("saudi arabia", "sa")
-    SINGAPORE = ("singapore", "sg")
+    SINGAPORE = ("singapore", "sg", "sg")
    SOUTHAFRICA = ("south africa", "za")
    SOUTHKOREA = ("south korea", "kr")
-    SPAIN = ("spain", "es")
+    SPAIN = ("spain", "es", "es")
    SWEDEN = ("sweden", "se")
-    SWITZERLAND = ("switzerland", "ch")
+    SWITZERLAND = ("switzerland", "ch", "de:ch")
    TAIWAN = ("taiwan", "tw")
    THAILAND = ("thailand", "th")
-    TURKEY = ("turkey", "tr")
+    TURKEY = ("türkiye,turkey", "tr")
    UKRAINE = ("ukraine", "ua")
    UNITEDARABEMIRATES = ("united arab emirates", "ae")
-    UK = ("uk", "uk")
-    USA = ("usa", "www")
+    UK = ("uk,united kingdom", "uk:gb", "co.uk")
+    USA = ("usa,us,united states", "www:us", "com")
    URUGUAY = ("uruguay", "uy")
    VENEZUELA = ("venezuela", "ve")
-    VIETNAM = ("vietnam", "vn")
+    VIETNAM = ("vietnam", "vn", "com")

    # internal for ziprecruiter
    US_CANADA = ("usa/ca", "www")

-    # internal for linkeind
+    # internal for linkedin
    WORLDWIDE = ("worldwide", "www")

-    def __new__(cls, country, domain):
-        obj = object.__new__(cls)
-        obj._value_ = country
-        obj.domain = domain
-        return obj
+    @property
+    def indeed_domain_value(self):
+        subdomain, _, api_country_code = self.value[1].partition(":")
+        if subdomain and api_country_code:
+            return subdomain, api_country_code.upper()
+        return self.value[1], self.value[1].upper()

    @property
-    def domain_value(self):
-        return self.domain
+    def glassdoor_domain_value(self):
+        if len(self.value) == 3:
+            subdomain, _, domain = self.value[2].partition(":")
+            if subdomain and domain:
+                return f"{subdomain}.glassdoor.{domain}"
+            else:
+                return f"www.glassdoor.{self.value[2]}"
+        else:
+            raise Exception(f"Glassdoor is not available for {self.name}")
+
+    def get_glassdoor_url(self):
+        return f"https://{self.glassdoor_domain_value}/"

    @classmethod
    def from_string(cls, country_str: str):
        """Convert a string to the corresponding Country enum."""
        country_str = country_str.strip().lower()
        for country in cls:
-            if country.value == country_str:
+            country_names = country.value[0].split(",")
+            if country_str in country_names:
                return country
        valid_countries = [country.value for country in cls]
        raise ValueError(
-            f"Invalid country string: '{country_str}'. Valid countries (only include this param for Indeed) are: {', '.join(valid_countries)}"
+            f"Invalid country string: '{country_str}'. Valid countries are: {', '.join([country[0] for country in valid_countries])}"
        )


 class Location(BaseModel):
-    country: Country = None
+    country: Country | str | None = None
    city: Optional[str] = None
    state: Optional[str] = None

@@ -153,11 +179,19 @@ class Location(BaseModel):
            location_parts.append(self.city)
        if self.state:
            location_parts.append(self.state)
-        if self.country and self.country not in (Country.US_CANADA, Country.WORLDWIDE):
-            if self.country.value in ("usa", "uk"):
-                location_parts.append(self.country.value.upper())
+        if isinstance(self.country, str):
+            location_parts.append(self.country)
+        elif self.country and self.country not in (
+            Country.US_CANADA,
+            Country.WORLDWIDE,
+        ):
+            country_name = self.country.value[0]
+            if "," in country_name:
+                country_name = country_name.split(",")[0]
+            if country_name in ("usa", "uk"):
+                location_parts.append(country_name.upper())
            else:
-                location_parts.append(self.country.value.title())
+                location_parts.append(country_name.title())
        return ", ".join(location_parts)


@@ -168,29 +202,65 @@ class CompensationInterval(Enum):
    DAILY = "daily"
    HOURLY = "hourly"

+    @classmethod
+    def get_interval(cls, pay_period):
+        interval_mapping = {
+            "YEAR": cls.YEARLY,
+            "HOUR": cls.HOURLY,
+        }
+        if pay_period in interval_mapping:
+            return interval_mapping[pay_period].value
+        else:
+            return cls[pay_period].value if pay_period in cls.__members__ else None
+

 class Compensation(BaseModel):
    interval: Optional[CompensationInterval] = None
-    min_amount: int = None
-    max_amount: int = None
+    min_amount: float | None = None
+    max_amount: float | None = None
    currency: Optional[str] = "USD"


+class DescriptionFormat(Enum):
+    MARKDOWN = "markdown"
+    HTML = "html"
+
+
 class JobPost(BaseModel):
+    id: str | None = None
    title: str
-    company_name: str
+    company_name: str | None
    job_url: str
+    job_url_direct: str | None = None
    location: Optional[Location]

    description: str | None = None
+    company_url: str | None = None
+    company_url_direct: str | None = None
+
    job_type: list[JobType] | None = None
    compensation: Compensation | None = None
    date_posted: date | None = None
-    benefits: str | None = None
    emails: list[str] | None = None
-    num_urgent_words: int | None = None
    is_remote: bool | None = None
-    # company_industry: str | None = None
+    listing_type: str | None = None
+
+    # linkedin specific
+    job_level: str | None = None
+
+    # linkedin and indeed specific
+    company_industry: str | None = None
+
+    # indeed specific
+    company_addresses: str | None = None
+    company_num_employees: str | None = None
+    company_revenue: str | None = None
+    company_description: str | None = None
+    logo_photo_url: str | None = None
+    banner_photo_url: str | None = None
+
+    # linkedin only atm
+    job_function: str | None = None


 class JobResponse(BaseModel):
--- a/src/jobspy/scrapers/init.py
+++ b/src/jobspy/scrapers/init.py
@@ -1,32 +1,51 @@
-from ..jobs import Enum, BaseModel, JobType, JobResponse, Country
-from typing import List, Optional, Any
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+
+from ..jobs import (
+    Enum,
+    BaseModel,
+    JobType,
+    JobResponse,
+    Country,
+    DescriptionFormat,
+)


 class Site(Enum):
    LINKEDIN = "linkedin"
    INDEED = "indeed"
    ZIP_RECRUITER = "zip_recruiter"
+    GLASSDOOR = "glassdoor"

+class SalarySource(Enum):
+    DIRECT_DATA = "direct_data"
+    DESCRIPTION = "description"

 class ScraperInput(BaseModel):
-    site_type: List[Site]
-    search_term: str
+    site_type: list[Site]
+    search_term: str | None = None

-    location: str = None
-    country: Optional[Country] = Country.USA
-    distance: Optional[int] = None
+    location: str | None = None
+    country: Country | None = Country.USA
+    distance: int | None = None
    is_remote: bool = False
-    job_type: Optional[JobType] = None
-    easy_apply: bool = None  # linkedin
+    job_type: JobType | None = None
+    easy_apply: bool | None = None
    offset: int = 0
+    linkedin_fetch_description: bool = False
+    linkedin_company_ids: list[int] | None = None
+    description_format: DescriptionFormat | None = DescriptionFormat.MARKDOWN

    results_wanted: int = 15
+    hours_old: int | None = None


-class Scraper:
-    def __init__(self, site: Site, proxy: Optional[List[str]] = None):
+class Scraper(ABC):
+    def __init__(self, site: Site, proxies: list[str] | None = None, ca_cert: str | None = None):
        self.site = site
-        self.proxy = (lambda p: {"http": p, "https": p} if p else None)(proxy)
+        self.proxies = proxies
+        self.ca_cert = ca_cert

-    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
-        ...
+    @abstractmethod
+    def scrape(self, scraper_input: ScraperInput) -> JobResponse: ...
--- a/src/jobspy/scrapers/exceptions.py
+++ b/src/jobspy/scrapers/exceptions.py
@@ -7,12 +7,20 @@ This module contains the set of Scrapers' exceptions.


 class LinkedInException(Exception):
-    """Failed to scrape LinkedIn"""
+    def __init__(self, message=None):
+        super().__init__(message or "An error occurred with LinkedIn")


 class IndeedException(Exception):
-    """Failed to scrape Indeed"""
+    def __init__(self, message=None):
+        super().__init__(message or "An error occurred with Indeed")


 class ZipRecruiterException(Exception):
-    """Failed to scrape ZipRecruiter"""
+    def __init__(self, message=None):
+        super().__init__(message or "An error occurred with ZipRecruiter")
+
+
+class GlassdoorException(Exception):
+    def __init__(self, message=None):
+        super().__init__(message or "An error occurred with Glassdoor")
--- a/src/jobspy/scrapers/glassdoor/init.py
+++ b/src/jobspy/scrapers/glassdoor/init.py
@@ -0,0 +1,364 @@
+"""
+jobspy.scrapers.glassdoor
+~~~~~~~~~~~~~~~~~~~
+
+This module contains routines to scrape Glassdoor.
+"""
+
+from __future__ import annotations
+
+import re
+import json
+import requests
+from typing import Optional, Tuple
+from datetime import datetime, timedelta
+from concurrent.futures import ThreadPoolExecutor, as_completed
+
+from .constants import fallback_token, query_template, headers
+from .. import Scraper, ScraperInput, Site
+from ..utils import extract_emails_from_text, create_logger
+from ..exceptions import GlassdoorException
+from ..utils import (
+    create_session,
+    markdown_converter,
+)
+from ...jobs import (
+    JobPost,
+    Compensation,
+    CompensationInterval,
+    Location,
+    JobResponse,
+    JobType,
+    DescriptionFormat,
+)
+
+logger = create_logger("Glassdoor")
+
+
+class GlassdoorScraper(Scraper):
+    def __init__(
+        self, proxies: list[str] | str | None = None, ca_cert: str | None = None
+    ):
+        """
+        Initializes GlassdoorScraper with the Glassdoor job search url
+        """
+        site = Site(Site.GLASSDOOR)
+        super().__init__(site, proxies=proxies, ca_cert=ca_cert)
+
+        self.base_url = None
+        self.country = None
+        self.session = None
+        self.scraper_input = None
+        self.jobs_per_page = 30
+        self.max_pages = 30
+        self.seen_urls = set()
+
+    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
+        """
+        Scrapes Glassdoor for jobs with scraper_input criteria.
+        :param scraper_input: Information about job search criteria.
+        :return: JobResponse containing a list of jobs.
+        """
+        self.scraper_input = scraper_input
+        self.scraper_input.results_wanted = min(900, scraper_input.results_wanted)
+        self.base_url = self.scraper_input.country.get_glassdoor_url()
+
+        self.session = create_session(
+            proxies=self.proxies, ca_cert=self.ca_cert, is_tls=True, has_retry=True
+        )
+        token = self._get_csrf_token()
+        headers["gd-csrf-token"] = token if token else fallback_token
+        self.session.headers.update(headers)
+
+        location_id, location_type = self._get_location(
+            scraper_input.location, scraper_input.is_remote
+        )
+        if location_type is None:
+            logger.error("Glassdoor: location not parsed")
+            return JobResponse(jobs=[])
+        job_list: list[JobPost] = []
+        cursor = None
+
+        range_start = 1 + (scraper_input.offset // self.jobs_per_page)
+        tot_pages = (scraper_input.results_wanted // self.jobs_per_page) + 2
+        range_end = min(tot_pages, self.max_pages + 1)
+        for page in range(range_start, range_end):
+            logger.info(f"search page: {page} / {range_end-1}")
+            try:
+                jobs, cursor = self._fetch_jobs_page(
+                    scraper_input, location_id, location_type, page, cursor
+                )
+                job_list.extend(jobs)
+                if not jobs or len(job_list) >= scraper_input.results_wanted:
+                    job_list = job_list[: scraper_input.results_wanted]
+                    break
+            except Exception as e:
+                logger.error(f"Glassdoor: {str(e)}")
+                break
+        return JobResponse(jobs=job_list)
+
+    def _fetch_jobs_page(
+        self,
+        scraper_input: ScraperInput,
+        location_id: int,
+        location_type: str,
+        page_num: int,
+        cursor: str | None,
+    ) -> Tuple[list[JobPost], str | None]:
+        """
+        Scrapes a page of Glassdoor for jobs with scraper_input criteria
+        """
+        jobs = []
+        self.scraper_input = scraper_input
+        try:
+            payload = self._add_payload(location_id, location_type, page_num, cursor)
+            response = self.session.post(
+                f"{self.base_url}/graph",
+                timeout_seconds=15,
+                data=payload,
+            )
+            if response.status_code != 200:
+                exc_msg = f"bad response status code: {response.status_code}"
+                raise GlassdoorException(exc_msg)
+            res_json = response.json()[0]
+            if "errors" in res_json:
+                raise ValueError("Error encountered in API response")
+        except (
+            requests.exceptions.ReadTimeout,
+            GlassdoorException,
+            ValueError,
+            Exception,
+        ) as e:
+            logger.error(f"Glassdoor: {str(e)}")
+            return jobs, None
+
+        jobs_data = res_json["data"]["jobListings"]["jobListings"]
+
+        with ThreadPoolExecutor(max_workers=self.jobs_per_page) as executor:
+            future_to_job_data = {
+                executor.submit(self._process_job, job): job for job in jobs_data
+            }
+            for future in as_completed(future_to_job_data):
+                try:
+                    job_post = future.result()
+                    if job_post:
+                        jobs.append(job_post)
+                except Exception as exc:
+                    raise GlassdoorException(f"Glassdoor generated an exception: {exc}")
+
+        return jobs, self.get_cursor_for_page(
+            res_json["data"]["jobListings"]["paginationCursors"], page_num + 1
+        )
+
+    def _get_csrf_token(self):
+        """
+        Fetches csrf token needed for API by visiting a generic page
+        """
+        res = self.session.get(f"{self.base_url}/Job/computer-science-jobs.htm")
+        pattern = r'"token":\s*"([^"]+)"'
+        matches = re.findall(pattern, res.text)
+        token = None
+        if matches:
+            token = matches[0]
+        return token
+
+    def _process_job(self, job_data):
+        """
+        Processes a single job and fetches its description.
+        """
+        job_id = job_data["jobview"]["job"]["listingId"]
+        job_url = f"{self.base_url}job-listing/j?jl={job_id}"
+        if job_url in self.seen_urls:
+            return None
+        self.seen_urls.add(job_url)
+        job = job_data["jobview"]
+        title = job["job"]["jobTitleText"]
+        company_name = job["header"]["employerNameFromSearch"]
+        company_id = job_data["jobview"]["header"]["employer"]["id"]
+        location_name = job["header"].get("locationName", "")
+        location_type = job["header"].get("locationType", "")
+        age_in_days = job["header"].get("ageInDays")
+        is_remote, location = False, None
+        date_diff = (datetime.now() - timedelta(days=age_in_days)).date()
+        date_posted = date_diff if age_in_days is not None else None
+
+        if location_type == "S":
+            is_remote = True
+        else:
+            location = self.parse_location(location_name)
+
+        compensation = self.parse_compensation(job["header"])
+        try:
+            description = self._fetch_job_description(job_id)
+        except:
+            description = None
+        company_url = f"{self.base_url}Overview/W-EI_IE{company_id}.htm"
+        company_logo = (
+            job_data["jobview"].get("overview", {}).get("squareLogoUrl", None)
+        )
+        listing_type = (
+            job_data["jobview"]
+            .get("header", {})
+            .get("adOrderSponsorshipLevel", "")
+            .lower()
+        )
+        return JobPost(
+            id=f"gd-{job_id}",
+            title=title,
+            company_url=company_url if company_id else None,
+            company_name=company_name,
+            date_posted=date_posted,
+            job_url=job_url,
+            location=location,
+            compensation=compensation,
+            is_remote=is_remote,
+            description=description,
+            emails=extract_emails_from_text(description) if description else None,
+            logo_photo_url=company_logo,
+            listing_type=listing_type,
+        )
+
+    def _fetch_job_description(self, job_id):
+        """
+        Fetches the job description for a single job ID.
+        """
+        url = f"{self.base_url}/graph"
+        body = [
+            {
+                "operationName": "JobDetailQuery",
+                "variables": {
+                    "jl": job_id,
+                    "queryString": "q",
+                    "pageTypeEnum": "SERP",
+                },
+                "query": """
+                query JobDetailQuery($jl: Long!, $queryString: String, $pageTypeEnum: PageTypeEnum) {
+                    jobview: jobView(
+                        listingId: $jl
+                        contextHolder: {queryString: $queryString, pageTypeEnum: $pageTypeEnum}
+                    ) {
+                        job {
+                            description
+                            __typename
+                        }
+                        __typename
+                    }
+                }
+                """,
+            }
+        ]
+        res = requests.post(url, json=body, headers=headers)
+        if res.status_code != 200:
+            return None
+        data = res.json()[0]
+        desc = data["data"]["jobview"]["job"]["description"]
+        if self.scraper_input.description_format == DescriptionFormat.MARKDOWN:
+            desc = markdown_converter(desc)
+        return desc
+
+    def _get_location(self, location: str, is_remote: bool) -> (int, str):
+        if not location or is_remote:
+            return "11047", "STATE"  # remote options
+        url = f"{self.base_url}/findPopularLocationAjax.htm?maxLocationsToReturn=10&term={location}"
+        res = self.session.get(url)
+        if res.status_code != 200:
+            if res.status_code == 429:
+                err = f"429 Response - Blocked by Glassdoor for too many requests"
+                logger.error(err)
+                return None, None
+            else:
+                err = f"Glassdoor response status code {res.status_code}"
+                err += f" - {res.text}"
+                logger.error(f"Glassdoor response status code {res.status_code}")
+                return None, None
+        items = res.json()
+
+        if not items:
+            raise ValueError(f"Location '{location}' not found on Glassdoor")
+        location_type = items[0]["locationType"]
+        if location_type == "C":
+            location_type = "CITY"
+        elif location_type == "S":
+            location_type = "STATE"
+        elif location_type == "N":
+            location_type = "COUNTRY"
+        return int(items[0]["locationId"]), location_type
+
+    def _add_payload(
+        self,
+        location_id: int,
+        location_type: str,
+        page_num: int,
+        cursor: str | None = None,
+    ) -> str:
+        fromage = None
+        if self.scraper_input.hours_old:
+            fromage = max(self.scraper_input.hours_old // 24, 1)
+        filter_params = []
+        if self.scraper_input.easy_apply:
+            filter_params.append({"filterKey": "applicationType", "values": "1"})
+        if fromage:
+            filter_params.append({"filterKey": "fromAge", "values": str(fromage)})
+        payload = {
+            "operationName": "JobSearchResultsQuery",
+            "variables": {
+                "excludeJobListingIds": [],
+                "filterParams": filter_params,
+                "keyword": self.scraper_input.search_term,
+                "numJobsToShow": 30,
+                "locationType": location_type,
+                "locationId": int(location_id),
+                "parameterUrlInput": f"IL.0,12_I{location_type}{location_id}",
+                "pageNumber": page_num,
+                "pageCursor": cursor,
+                "fromage": fromage,
+                "sort": "date",
+            },
+            "query": query_template,
+        }
+        if self.scraper_input.job_type:
+            payload["variables"]["filterParams"].append(
+                {"filterKey": "jobType", "values": self.scraper_input.job_type.value[0]}
+            )
+        return json.dumps([payload])
+
+    @staticmethod
+    def parse_compensation(data: dict) -> Optional[Compensation]:
+        pay_period = data.get("payPeriod")
+        adjusted_pay = data.get("payPeriodAdjustedPay")
+        currency = data.get("payCurrency", "USD")
+        if not pay_period or not adjusted_pay:
+            return None
+
+        interval = None
+        if pay_period == "ANNUAL":
+            interval = CompensationInterval.YEARLY
+        elif pay_period:
+            interval = CompensationInterval.get_interval(pay_period)
+        min_amount = int(adjusted_pay.get("p10") // 1)
+        max_amount = int(adjusted_pay.get("p90") // 1)
+        return Compensation(
+            interval=interval,
+            min_amount=min_amount,
+            max_amount=max_amount,
+            currency=currency,
+        )
+
+    @staticmethod
+    def get_job_type_enum(job_type_str: str) -> list[JobType] | None:
+        for job_type in JobType:
+            if job_type_str in job_type.value:
+                return [job_type]
+
+    @staticmethod
+    def parse_location(location_name: str) -> Location | None:
+        if not location_name or location_name == "Remote":
+            return
+        city, _, state = location_name.partition(", ")
+        return Location(city=city, state=state)
+
+    @staticmethod
+    def get_cursor_for_page(pagination_cursors, page_num):
+        for cursor_data in pagination_cursors:
+            if cursor_data["pageNumber"] == page_num:
+                return cursor_data["cursor"]
--- a/src/jobspy/scrapers/glassdoor/constants.py
+++ b/src/jobspy/scrapers/glassdoor/constants.py
@@ -0,0 +1,184 @@
+headers = {
+    "authority": "www.glassdoor.com",
+    "accept": "*/*",
+    "accept-language": "en-US,en;q=0.9",
+    "apollographql-client-name": "job-search-next",
+    "apollographql-client-version": "4.65.5",
+    "content-type": "application/json",
+    "origin": "https://www.glassdoor.com",
+    "referer": "https://www.glassdoor.com/",
+    "sec-ch-ua": '"Chromium";v="118", "Google Chrome";v="118", "Not=A?Brand";v="99"',
+    "sec-ch-ua-mobile": "?0",
+    "sec-ch-ua-platform": '"macOS"',
+    "sec-fetch-dest": "empty",
+    "sec-fetch-mode": "cors",
+    "sec-fetch-site": "same-origin",
+    "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36",
+}
+query_template = """
+        query JobSearchResultsQuery(
+            $excludeJobListingIds: [Long!], 
+            $keyword: String, 
+            $locationId: Int, 
+            $locationType: LocationTypeEnum, 
+            $numJobsToShow: Int!, 
+            $pageCursor: String, 
+            $pageNumber: Int, 
+            $filterParams: [FilterParams], 
+            $originalPageUrl: String, 
+            $seoFriendlyUrlInput: String, 
+            $parameterUrlInput: String, 
+            $seoUrl: Boolean
+        ) {
+            jobListings(
+                contextHolder: {
+                    searchParams: {
+                        excludeJobListingIds: $excludeJobListingIds, 
+                        keyword: $keyword, 
+                        locationId: $locationId, 
+                        locationType: $locationType, 
+                        numPerPage: $numJobsToShow, 
+                        pageCursor: $pageCursor, 
+                        pageNumber: $pageNumber, 
+                        filterParams: $filterParams, 
+                        originalPageUrl: $originalPageUrl, 
+                        seoFriendlyUrlInput: $seoFriendlyUrlInput, 
+                        parameterUrlInput: $parameterUrlInput, 
+                        seoUrl: $seoUrl, 
+                        searchType: SR
+                    }
+                }
+            ) {
+                companyFilterOptions {
+                    id
+                    shortName
+                    __typename
+                }
+                filterOptions
+                indeedCtk
+                jobListings {
+                    ...JobView
+                    __typename
+                }
+                jobListingSeoLinks {
+                    linkItems {
+                        position
+                        url
+                        __typename
+                    }
+                    __typename
+                }
+                jobSearchTrackingKey
+                jobsPageSeoData {
+                    pageMetaDescription
+                    pageTitle
+                    __typename
+                }
+                paginationCursors {
+                    cursor
+                    pageNumber
+                    __typename
+                }
+                indexablePageForSeo
+                searchResultsMetadata {
+                    searchCriteria {
+                        implicitLocation {
+                            id
+                            localizedDisplayName
+                            type
+                            __typename
+                        }
+                        keyword
+                        location {
+                            id
+                            shortName
+                            localizedShortName
+                            localizedDisplayName
+                            type
+                            __typename
+                        }
+                        __typename
+                    }
+                    helpCenterDomain
+                    helpCenterLocale
+                    jobSerpJobOutlook {
+                        occupation
+                        paragraph
+                        __typename
+                    }
+                    showMachineReadableJobs
+                    __typename
+                }
+                totalJobsCount
+                __typename
+            }
+        }
+
+        fragment JobView on JobListingSearchResult {
+            jobview {
+                header {
+                    adOrderId
+                    advertiserType
+                    adOrderSponsorshipLevel
+                    ageInDays
+                    divisionEmployerName
+                    easyApply
+                    employer {
+                        id
+                        name
+                        shortName
+                        __typename
+                    }
+                    employerNameFromSearch
+                    goc
+                    gocConfidence
+                    gocId
+                    jobCountryId
+                    jobLink
+                    jobResultTrackingKey
+                    jobTitleText
+                    locationName
+                    locationType
+                    locId
+                    needsCommission
+                    payCurrency
+                    payPeriod
+                    payPeriodAdjustedPay {
+                        p10
+                        p50
+                        p90
+                        __typename
+                    }
+                    rating
+                    salarySource
+                    savedJobId
+                    sponsored
+                    __typename
+                }
+                job {
+                    description
+                    importConfigId
+                    jobTitleId
+                    jobTitleText
+                    listingId
+                    __typename
+                }
+                jobListingAdminDetails {
+                    cpcVal
+                    importConfigId
+                    jobListingId
+                    jobSourceId
+                    userEligibleForAdminJobDetails
+                    __typename
+                }
+                overview {
+                    shortName
+                    squareLogoUrl
+                    __typename
+                }
+                __typename
+            }
+            __typename
+        }
+"""
+fallback_token = "Ft6oHEWlRZrxDww95Cpazw:0pGUrkb2y3TyOpAIqF2vbPmUXoXVkD3oEGDVkvfeCerceQ5-n8mBg3BovySUIjmCPHCaW0H2nQVdqzbtsYqf4Q:wcqRqeegRUa9MVLJGyujVXB7vWFPjdaS1CtrrzJq-ok"
--- a/src/jobspy/scrapers/indeed/init.py
+++ b/src/jobspy/scrapers/indeed/init.py
@@ -4,19 +4,22 @@ jobspy.scrapers.indeed

 This module contains routines to scrape Indeed.
 """
-import re
+
+from __future__ import annotations
+
 import math
-import io
-import json
+from typing import Tuple
 from datetime import datetime

-import urllib.parse
-from bs4 import BeautifulSoup
-from bs4.element import Tag
-from concurrent.futures import ThreadPoolExecutor, Future
-
-from ..exceptions import IndeedException
-from ..utils import count_urgent_words, extract_emails_from_text, create_session
+from .constants import job_search_query, api_headers
+from .. import Scraper, ScraperInput, Site
+from ..utils import (
+    extract_emails_from_text,
+    get_enum_from_job_type,
+    markdown_converter,
+    create_session,
+    create_logger,
+)
 from ...jobs import (
    JobPost,
    Compensation,
@@ -24,153 +27,32 @@ from ...jobs import (
    Location,
    JobResponse,
    JobType,
+    DescriptionFormat,
 )
-from .. import Scraper, ScraperInput, Site
+
+logger = create_logger("Indeed")


 class IndeedScraper(Scraper):
-    def __init__(self, proxy: str | None = None):
-        """
-        Initializes IndeedScraper with the Indeed job search url
-        """
-        self.url = None
-        self.country = None
-        site = Site(Site.INDEED)
-        super().__init__(site, proxy=proxy)
-
-        self.jobs_per_page = 15
-        self.seen_urls = set()
-
-    def scrape_page(
-        self, scraper_input: ScraperInput, page: int
-    ) -> tuple[list[JobPost], int]:
-        """
-        Scrapes a page of Indeed for jobs with scraper_input criteria
-        :param scraper_input:
-        :param page:
-        :return: jobs found on page, total number of jobs found for search
-        """
-        self.country = scraper_input.country
-        domain = self.country.domain_value
-        self.url = f"https://{domain}.indeed.com"
-        session = create_session(self.proxy)
-
-        params = {
-            "q": scraper_input.search_term,
-            "l": scraper_input.location,
-            "filter": 0,
-            "start": scraper_input.offset + page * 10,
-        }
-        if scraper_input.distance:
-            params["radius"] = scraper_input.distance
-
-        sc_values = []
-        if scraper_input.is_remote:
-            sc_values.append("attr(DSQF7)")
-        if scraper_input.job_type:
-            sc_values.append("jt({})".format(scraper_input.job_type.value))
-
-        if sc_values:
-            params["sc"] = "0kf:" + "".join(sc_values) + ";"
-        try:
-            response = session.get(
-                f"{self.url}/jobs",
-                headers=self.get_headers(),
-                params=params,
-                allow_redirects=True,
-                timeout_seconds=10,
-            )
-            if response.status_code not in range(200, 400):
-                raise IndeedException(
-                    f"bad response with status code: {response.status_code}"
-                )
-        except Exception as e:
-            if "Proxy responded with" in str(e):
-                raise IndeedException("bad proxy")
-            raise IndeedException(str(e))
-
-        soup = BeautifulSoup(response.content, "html.parser")
-        if "did not match any jobs" in response.text:
-            raise IndeedException("Parsing exception: Search did not match any jobs")
-
-        jobs = IndeedScraper.parse_jobs(
-            soup
-        )  #: can raise exception, handled by main scrape function
-        total_num_jobs = IndeedScraper.total_jobs(soup)
-
-        if (
-            not jobs.get("metaData", {})
-            .get("mosaicProviderJobCardsModel", {})
-            .get("results")
+    def __init__(
+        self, proxies: list[str] | str | None = None, ca_cert: str | None = None
    ):
-            raise IndeedException("No jobs found.")
+        """
+        Initializes IndeedScraper with the Indeed API url
+        """
+        super().__init__(Site.INDEED, proxies=proxies)

-        def process_job(job) -> JobPost | None:
-            job_url = f'{self.url}/jobs/viewjob?jk={job["jobkey"]}'
-            job_url_client = f'{self.url}/viewjob?jk={job["jobkey"]}'
-            if job_url in self.seen_urls:
-                return None
-
-            extracted_salary = job.get("extractedSalary")
-            compensation = None
-            if extracted_salary:
-                salary_snippet = job.get("salarySnippet")
-                currency = salary_snippet.get("currency") if salary_snippet else None
-                interval = (extracted_salary.get("type"),)
-                if isinstance(interval, tuple):
-                    interval = interval[0]
-
-                interval = interval.upper()
-                if interval in CompensationInterval.__members__:
-                    compensation = Compensation(
-                        interval=CompensationInterval[interval],
-                        min_amount=int(extracted_salary.get("min")),
-                        max_amount=int(extracted_salary.get("max")),
-                        currency=currency,
+        self.session = create_session(
+            proxies=self.proxies, ca_cert=ca_cert, is_tls=False
        )
-
-            job_type = IndeedScraper.get_job_type(job)
-            timestamp_seconds = job["pubDate"] / 1000
-            date_posted = datetime.fromtimestamp(timestamp_seconds)
-            date_posted = date_posted.strftime("%Y-%m-%d")
-
-            description = self.get_description(job_url)
-            with io.StringIO(job["snippet"]) as f:
-                soup_io = BeautifulSoup(f, "html.parser")
-                li_elements = soup_io.find_all("li")
-                if description is None and li_elements:
-                    description = " ".join(li.text for li in li_elements)
-
-            job_post = JobPost(
-                title=job["normTitle"],
-                description=description,
-                company_name=job["company"],
-                location=Location(
-                    city=job.get("jobLocationCity"),
-                    state=job.get("jobLocationState"),
-                    country=self.country,
-                ),
-                job_type=job_type,
-                compensation=compensation,
-                date_posted=date_posted,
-                job_url=job_url_client,
-                emails=extract_emails_from_text(description) if description else None,
-                num_urgent_words=count_urgent_words(description)
-                if description
-                else None,
-                is_remote=self.is_remote_job(job),
-            )
-            return job_post
-
-        with ThreadPoolExecutor(max_workers=1) as executor:
-            job_results: list[Future] = [
-                executor.submit(process_job, job)
-                for job in jobs["metaData"]["mosaicProviderJobCardsModel"]["results"]
-            ]
-
-        job_list = [result.result() for result in job_results if result.result()]
-
-        return job_list, total_num_jobs
+        self.scraper_input = None
+        self.jobs_per_page = 100
+        self.num_workers = 10
+        self.seen_urls = set()
+        self.headers = None
+        self.api_country_code = None
+        self.base_url = None
+        self.api_url = "https://apis.indeed.com/graphql"

    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
        """
@@ -178,178 +60,290 @@ class IndeedScraper(Scraper):
        :param scraper_input:
        :return: job_response
        """
-        pages_to_process = (
-            math.ceil(scraper_input.results_wanted / self.jobs_per_page) - 1
+        self.scraper_input = scraper_input
+        domain, self.api_country_code = self.scraper_input.country.indeed_domain_value
+        self.base_url = f"https://{domain}.indeed.com"
+        self.headers = api_headers.copy()
+        self.headers["indeed-co"] = self.scraper_input.country.indeed_domain_value
+        job_list = []
+        page = 1
+
+        cursor = None
+
+        while len(self.seen_urls) < scraper_input.results_wanted + scraper_input.offset:
+            logger.info(
+                f"search page: {page} / {math.ceil(scraper_input.results_wanted / 100)}"
            )
-
-        #: get first page to initialize session
-        job_list, total_results = self.scrape_page(scraper_input, 0)
-
-        with ThreadPoolExecutor(max_workers=1) as executor:
-            futures: list[Future] = [
-                executor.submit(self.scrape_page, scraper_input, page)
-                for page in range(1, pages_to_process + 1)
-            ]
-
-            for future in futures:
-                jobs, _ = future.result()
-
+            jobs, cursor = self._scrape_page(cursor)
+            if not jobs:
+                logger.info(f"found no jobs on page: {page}")
+                break
            job_list += jobs
-
-        if len(job_list) > scraper_input.results_wanted:
-            job_list = job_list[: scraper_input.results_wanted]
-
-        job_response = JobResponse(
-            jobs=job_list,
-            total_results=total_results,
+            page += 1
+        return JobResponse(
+            jobs=job_list[
+                scraper_input.offset : scraper_input.offset
+                + scraper_input.results_wanted
+            ]
        )
-        return job_response

-    def get_description(self, job_page_url: str) -> str | None:
+    def _scrape_page(self, cursor: str | None) -> Tuple[list[JobPost], str | None]:
        """
-        Retrieves job description by going to the job page url
-        :param job_page_url:
-        :return: description
+        Scrapes a page of Indeed for jobs with scraper_input criteria
+        :param cursor:
+        :return: jobs found on page, next page cursor
        """
-        parsed_url = urllib.parse.urlparse(job_page_url)
-        params = urllib.parse.parse_qs(parsed_url.query)
-        jk_value = params.get("jk", [None])[0]
-        formatted_url = f"{self.url}/viewjob?jk={jk_value}&spa=1"
-        session = create_session(self.proxy)
-
-        try:
-            response = session.get(
-                formatted_url,
-                headers=self.get_headers(),
-                allow_redirects=True,
-                timeout_seconds=5,
+        jobs = []
+        new_cursor = None
+        filters = self._build_filters()
+        search_term = (
+            self.scraper_input.search_term.replace('"', '\\"')
+            if self.scraper_input.search_term
+            else ""
        )
-        except Exception as e:
-            return None
+        query = job_search_query.format(
+            what=(f'what: "{search_term}"' if search_term else ""),
+            location=(
+                f'location: {{where: "{self.scraper_input.location}", radius: {self.scraper_input.distance}, radiusUnit: MILES}}'
+                if self.scraper_input.location
+                else ""
+            ),
+            dateOnIndeed=self.scraper_input.hours_old,
+            cursor=f'cursor: "{cursor}"' if cursor else "",
+            filters=filters,
+        )
+        payload = {
+            "query": query,
+        }
+        api_headers_temp = api_headers.copy()
+        api_headers_temp["indeed-co"] = self.api_country_code
+        response = self.session.post(
+            self.api_url,
+            headers=api_headers_temp,
+            json=payload,
+            timeout=10,
+        )
+        if not response.ok:
+            logger.info(
+                f"responded with status code: {response.status_code} (submit GitHub issue if this appears to be a bug)"
+            )
+            return jobs, new_cursor
+        data = response.json()
+        jobs = data["data"]["jobSearch"]["results"]
+        new_cursor = data["data"]["jobSearch"]["pageInfo"]["nextCursor"]

-        if response.status_code not in range(200, 400):
-            return None
+        job_list = []
+        for job in jobs:
+            processed_job = self._process_job(job["job"])
+            if processed_job:
+                job_list.append(processed_job)

-        raw_description = response.json()["body"]["jobInfoWrapperModel"][
-            "jobInfoModel"
-        ]["sanitizedJobDescription"]
-        with io.StringIO(raw_description) as f:
-            soup = BeautifulSoup(f, "html.parser")
-            text_content = " ".join(soup.get_text().split()).strip()
-            return text_content
+        return job_list, new_cursor
+
+    def _build_filters(self):
+        """
+        Builds the filters dict for job type/is_remote. If hours_old is provided, composite filter for job_type/is_remote is not possible.
+        IndeedApply: filters: { keyword: { field: "indeedApplyScope", keys: ["DESKTOP"] } }
+        """
+        filters_str = ""
+        if self.scraper_input.hours_old:
+            filters_str = """
+            filters: {{
+                date: {{
+                  field: "dateOnIndeed",
+                  start: "{start}h"
+                }}
+            }}
+            """.format(
+                start=self.scraper_input.hours_old
+            )
+        elif self.scraper_input.easy_apply:
+            filters_str = """
+            filters: {
+                keyword: {
+                  field: "indeedApplyScope",
+                  keys: ["DESKTOP"]
+                }
+            }
+            """
+        elif self.scraper_input.job_type or self.scraper_input.is_remote:
+            job_type_key_mapping = {
+                JobType.FULL_TIME: "CF3CP",
+                JobType.PART_TIME: "75GKK",
+                JobType.CONTRACT: "NJXCK",
+                JobType.INTERNSHIP: "VDTG7",
+            }
+
+            keys = []
+            if self.scraper_input.job_type:
+                key = job_type_key_mapping[self.scraper_input.job_type]
+                keys.append(key)
+
+            if self.scraper_input.is_remote:
+                keys.append("DSQF7")
+
+            if keys:
+                keys_str = '", "'.join(keys)
+                filters_str = f"""
+                filters: {{
+                  composite: {{
+                    filters: [{{
+                      keyword: {{
+                        field: "attributes",
+                        keys: ["{keys_str}"]
+                      }}
+                    }}]
+                  }}
+                }}
+                """
+        return filters_str
+
+    def _process_job(self, job: dict) -> JobPost | None:
+        """
+        Parses the job dict into JobPost model
+        :param job: dict to parse
+        :return: JobPost if it's a new job
+        """
+        job_url = f'{self.base_url}/viewjob?jk={job["key"]}'
+        if job_url in self.seen_urls:
+            return
+        self.seen_urls.add(job_url)
+        description = job["description"]["html"]
+        if self.scraper_input.description_format == DescriptionFormat.MARKDOWN:
+            description = markdown_converter(description)
+
+        job_type = self._get_job_type(job["attributes"])
+        timestamp_seconds = job["datePublished"] / 1000
+        date_posted = datetime.fromtimestamp(timestamp_seconds).strftime("%Y-%m-%d")
+        employer = job["employer"].get("dossier") if job["employer"] else None
+        employer_details = employer.get("employerDetails", {}) if employer else {}
+        rel_url = job["employer"]["relativeCompanyPageUrl"] if job["employer"] else None
+        return JobPost(
+            id=f'in-{job["key"]}',
+            title=job["title"],
+            description=description,
+            company_name=job["employer"].get("name") if job.get("employer") else None,
+            company_url=(f"{self.base_url}{rel_url}" if job["employer"] else None),
+            company_url_direct=(
+                employer["links"]["corporateWebsite"] if employer else None
+            ),
+            location=Location(
+                city=job.get("location", {}).get("city"),
+                state=job.get("location", {}).get("admin1Code"),
+                country=job.get("location", {}).get("countryCode"),
+            ),
+            job_type=job_type,
+            compensation=self._get_compensation(job["compensation"]),
+            date_posted=date_posted,
+            job_url=job_url,
+            job_url_direct=(
+                job["recruit"].get("viewJobUrl") if job.get("recruit") else None
+            ),
+            emails=extract_emails_from_text(description) if description else None,
+            is_remote=self._is_job_remote(job, description),
+            company_addresses=(
+                employer_details["addresses"][0]
+                if employer_details.get("addresses")
+                else None
+            ),
+            company_industry=(
+                employer_details["industry"]
+                .replace("Iv1", "")
+                .replace("_", " ")
+                .title()
+                .strip()
+                if employer_details.get("industry")
+                else None
+            ),
+            company_num_employees=employer_details.get("employeesLocalizedLabel"),
+            company_revenue=employer_details.get("revenueLocalizedLabel"),
+            company_description=employer_details.get("briefDescription"),
+            logo_photo_url=(
+                employer["images"].get("squareLogoUrl")
+                if employer and employer.get("images")
+                else None
+            ),
+        )

    @staticmethod
-    def get_job_type(job: dict) -> list[JobType] | None:
+    def _get_job_type(attributes: list) -> list[JobType]:
        """
-        Parses the job to get list of job types
-        :param job:
-        :return:
+        Parses the attributes to get list of job types
+        :param attributes:
+        :return: list of JobType
        """
        job_types: list[JobType] = []
-        for taxonomy in job["taxonomyAttributes"]:
-            if taxonomy["label"] == "job-types":
-                for i in range(len(taxonomy["attributes"])):
-                    label = taxonomy["attributes"][i].get("label")
-                    if label:
-                        job_type_str = label.replace("-", "").replace(" ", "").lower()
-                        job_types.append(
-                            IndeedScraper.get_enum_from_job_type(job_type_str)
-                        )
+        for attribute in attributes:
+            job_type_str = attribute["label"].replace("-", "").replace(" ", "").lower()
+            job_type = get_enum_from_job_type(job_type_str)
+            if job_type:
+                job_types.append(job_type)
        return job_types

    @staticmethod
-    def get_enum_from_job_type(job_type_str):
+    def _get_compensation(compensation: dict) -> Compensation | None:
        """
-        Given a string, returns the corresponding JobType enum member if a match is found.
-        for job_type in JobType:
+        Parses the job to get compensation
+        :param job:
+        :return: compensation object
        """
-        for job_type in JobType:
-            if job_type_str in job_type.value:
-                return job_type
+        if not compensation["baseSalary"] and not compensation["estimated"]:
            return None
-
-    @staticmethod
-    def parse_jobs(soup: BeautifulSoup) -> dict:
-        """
-        Parses the jobs from the soup object
-        :param soup:
-        :return: jobs
-        """
-
-        def find_mosaic_script() -> Tag | None:
-            """
-            Finds jobcards script tag
-            :return: script_tag
-            """
-            script_tags = soup.find_all("script")
-
-            for tag in script_tags:
-                if (
-                    tag.string
-                    and "mosaic.providerData" in tag.string
-                    and "mosaic-provider-jobcards" in tag.string
-                ):
-                    return tag
+        comp = (
+            compensation["baseSalary"]
+            if compensation["baseSalary"]
+            else compensation["estimated"]["baseSalary"]
+        )
+        if not comp:
            return None
-
-        script_tag = find_mosaic_script()
-
-        if script_tag:
-            script_str = script_tag.string
-            pattern = r'window.mosaic.providerData\["mosaic-provider-jobcards"\]\s*=\s*({.*?});'
-            p = re.compile(pattern, re.DOTALL)
-            m = p.search(script_str)
-            if m:
-                jobs = json.loads(m.group(1).strip())
-                return jobs
-            else:
-                raise IndeedException("Could not find mosaic provider job cards data")
-        else:
-            raise IndeedException(
-                "Could not find a script tag containing mosaic provider data"
+        interval = IndeedScraper._get_compensation_interval(comp["unitOfWork"])
+        if not interval:
+            return None
+        min_range = comp["range"].get("min")
+        max_range = comp["range"].get("max")
+        return Compensation(
+            interval=interval,
+            min_amount=int(min_range) if min_range is not None else None,
+            max_amount=int(max_range) if max_range is not None else None,
+            currency=(
+                compensation["estimated"]["currencyCode"]
+                if compensation["estimated"]
+                else compensation["currencyCode"]
+            ),
        )

    @staticmethod
-    def total_jobs(soup: BeautifulSoup) -> int:
+    def _is_job_remote(job: dict, description: str) -> bool:
        """
-        Parses the total jobs for that search from soup object
-        :param soup:
-        :return: total_num_jobs
+        Searches the description, location, and attributes to check if job is remote
        """
-        script = soup.find("script", string=lambda t: t and "window._initialData" in t)
-
-        pattern = re.compile(r"window._initialData\s*=\s*({.*})\s*;", re.DOTALL)
-        match = pattern.search(script.string)
-        total_num_jobs = 0
-        if match:
-            json_str = match.group(1)
-            data = json.loads(json_str)
-            total_num_jobs = int(data["searchTitleBarModel"]["totalNumResults"])
-        return total_num_jobs
+        remote_keywords = ["remote", "work from home", "wfh"]
+        is_remote_in_attributes = any(
+            any(keyword in attr["label"].lower() for keyword in remote_keywords)
+            for attr in job["attributes"]
+        )
+        is_remote_in_description = any(
+            keyword in description.lower() for keyword in remote_keywords
+        )
+        is_remote_in_location = any(
+            keyword in job["location"]["formatted"]["long"].lower()
+            for keyword in remote_keywords
+        )
+        return (
+            is_remote_in_attributes or is_remote_in_description or is_remote_in_location
+        )

    @staticmethod
-    def get_headers():
-        return {
-            "authority": "www.indeed.com",
-            "accept": "*/*",
-            "accept-language": "en-US,en;q=0.9",
-            "referer": "https://www.indeed.com/viewjob?jk=fe6182337d72c7b1&tk=1hcbfcmd0k62t802&from=serp&vjs=3&advn=8132938064490989&adid=408692607&ad=-6NYlbfkN0A3Osc99MJFDKjquSk4WOGT28ALb_ad4QMtrHreCb9ICg6MiSVy9oDAp3evvOrI7Q-O9qOtQTg1EPbthP9xWtBN2cOuVeHQijxHjHpJC65TjDtftH3AXeINjBvAyDrE8DrRaAXl8LD3Fs1e_xuDHQIssdZ2Mlzcav8m5jHrA0fA64ZaqJV77myldaNlM7-qyQpy4AsJQfvg9iR2MY7qeC5_FnjIgjKIy_lNi9OPMOjGRWXA94CuvC7zC6WeiJmBQCHISl8IOBxf7EdJZlYdtzgae3593TFxbkd6LUwbijAfjax39aAuuCXy3s9C4YgcEP3TwEFGQoTpYu9Pmle-Ae1tHGPgsjxwXkgMm7Cz5mBBdJioglRCj9pssn-1u1blHZM4uL1nK9p1Y6HoFgPUU9xvKQTHjKGdH8d4y4ETyCMoNF4hAIyUaysCKdJKitC8PXoYaWhDqFtSMR4Jys8UPqUV&xkcb=SoDD-_M3JLQfWnQTDh0LbzkdCdPP&xpse=SoBa6_I3JLW9FlWZlB0PbzkdCdPP&sjdu=i6xVERweJM_pVUvgf-MzuaunBTY7G71J5eEX6t4DrDs5EMPQdODrX7Nn-WIPMezoqr5wA_l7Of-3CtoiUawcHw",
-            "sec-ch-ua": '"Google Chrome";v="119", "Chromium";v="119", "Not?A_Brand";v="24"',
-            "sec-ch-ua-mobile": "?0",
-            "sec-ch-ua-platform": '"Windows"',
-            "sec-fetch-dest": "empty",
-            "sec-fetch-mode": "cors",
-            "sec-fetch-site": "same-origin",
-            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36",
+    def _get_compensation_interval(interval: str) -> CompensationInterval:
+        interval_mapping = {
+            "DAY": "DAILY",
+            "YEAR": "YEARLY",
+            "HOUR": "HOURLY",
+            "WEEK": "WEEKLY",
+            "MONTH": "MONTHLY",
        }
-
-    @staticmethod
-    def is_remote_job(job: dict) -> bool:
-        """
-        :param job:
-        :return: bool
-        """
-        for taxonomy in job.get("taxonomyAttributes", []):
-            if taxonomy["label"] == "remote" and len(taxonomy["attributes"]) > 0:
-                return True
-        return False
+        mapped_interval = interval_mapping.get(interval.upper(), None)
+        if mapped_interval and mapped_interval in CompensationInterval.__members__:
+            return CompensationInterval[mapped_interval]
+        else:
+            raise ValueError(f"Unsupported interval: {interval}")
--- a/src/jobspy/scrapers/indeed/constants.py
+++ b/src/jobspy/scrapers/indeed/constants.py
@@ -0,0 +1,109 @@
+job_search_query = """
+    query GetJobData {{
+        jobSearch(
+        {what}
+        {location}
+        limit: 100
+        {cursor}
+        sort: RELEVANCE
+        {filters}
+        ) {{
+        pageInfo {{
+            nextCursor
+        }}
+        results {{
+            trackingKey
+            job {{
+            source {{
+                name
+            }}
+            key
+            title
+            datePublished
+            dateOnIndeed
+            description {{
+                html
+            }}
+            location {{
+                countryName
+                countryCode
+                admin1Code
+                city
+                postalCode
+                streetAddress
+                formatted {{
+                short
+                long
+                }}
+            }}
+            compensation {{
+                estimated {{
+                currencyCode
+                baseSalary {{
+                    unitOfWork
+                    range {{
+                    ... on Range {{
+                        min
+                        max
+                    }}
+                    }}
+                }}
+                }}
+                baseSalary {{
+                unitOfWork
+                range {{
+                    ... on Range {{
+                    min
+                    max
+                    }}
+                }}
+                }}
+                currencyCode
+            }}
+            attributes {{
+                key
+                label
+            }}
+            employer {{
+                relativeCompanyPageUrl
+                name
+                dossier {{
+                    employerDetails {{
+                    addresses
+                    industry
+                    employeesLocalizedLabel
+                    revenueLocalizedLabel
+                    briefDescription
+                    ceoName
+                    ceoPhotoUrl
+                    }}
+                    images {{
+                        headerImageUrl
+                        squareLogoUrl
+                    }}
+                    links {{
+                    corporateWebsite
+                }}
+                }}
+            }}
+            recruit {{
+                viewJobUrl
+                detailedSalary
+                workSchedule
+            }}
+            }}
+        }}
+        }}
+    }}
+    """
+
+api_headers = {
+    "Host": "apis.indeed.com",
+    "content-type": "application/json",
+    "indeed-api-key": "161092c2017b5bbab13edb12461a62d5a833871e7cad6d9d475304573de67ac8",
+    "accept": "application/json",
+    "indeed-locale": "en-US",
+    "accept-language": "en-US,en;q=0.9",
+    "user-agent": "Mozilla/5.0 (iPhone; CPU iPhone OS 16_6_1 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 Indeed App 193.1",
+    "indeed-app-info": "appv=193.1; appid=com.indeed.jobsearch; osv=16.6.1; os=ios; dtype=phone",
+}
--- a/src/jobspy/scrapers/linkedin/init.py
+++ b/src/jobspy/scrapers/linkedin/init.py
@@ -4,41 +4,68 @@ jobspy.scrapers.linkedin

 This module contains routines to scrape LinkedIn.
 """
+
+from __future__ import annotations
+
+import math
+import time
+import random
+import regex as re
 from typing import Optional
 from datetime import datetime

-import requests
-import time
-import re
-from requests.exceptions import ProxyError
-from concurrent.futures import ThreadPoolExecutor, as_completed
-from bs4 import BeautifulSoup
 from bs4.element import Tag
-from threading import Lock
+from bs4 import BeautifulSoup
+from urllib.parse import urlparse, urlunparse, unquote

+from .constants import headers
 from .. import Scraper, ScraperInput, Site
-from ..utils import count_urgent_words, extract_emails_from_text
 from ..exceptions import LinkedInException
+from ..utils import create_session, remove_attributes, create_logger
 from ...jobs import (
    JobPost,
    Location,
    JobResponse,
    JobType,
+    Country,
+    Compensation,
+    DescriptionFormat,
 )
+from ..utils import (
+    extract_emails_from_text,
+    get_enum_from_job_type,
+    currency_parser,
+    markdown_converter,
+)
+
+logger = create_logger("LinkedIn")


 class LinkedInScraper(Scraper):
-    MAX_RETRIES = 3
-    DELAY = 10
+    base_url = "https://www.linkedin.com"
+    delay = 3
+    band_delay = 4
+    jobs_per_page = 25

-    def __init__(self, proxy: Optional[str] = None):
+    def __init__(
+        self, proxies: list[str] | str | None = None, ca_cert: str | None = None
+    ):
        """
        Initializes LinkedInScraper with the LinkedIn job search url
        """
-        site = Site(Site.LINKEDIN)
+        super().__init__(Site.LINKEDIN, proxies=proxies, ca_cert=ca_cert)
+        self.session = create_session(
+            proxies=self.proxies,
+            ca_cert=ca_cert,
+            is_tls=False,
+            has_retry=True,
+            delay=5,
+            clear_cookies=True,
+        )
+        self.session.headers.update(headers)
+        self.scraper_input = None
        self.country = "worldwide"
-        self.url = "https://www.linkedin.com"
-        super().__init__(site, proxy=proxy)
+        self.job_url_direct_regex = re.compile(r'(?<=\?url=)[^"]+')

    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
        """
@@ -46,117 +73,133 @@ class LinkedInScraper(Scraper):
        :param scraper_input:
        :return: job_response
        """
+        self.scraper_input = scraper_input
        job_list: list[JobPost] = []
-        seen_urls = set()
-        url_lock = Lock()
-        page = scraper_input.offset // 25 + 25 if scraper_input.offset else 0
-
-        def job_type_code(job_type_enum):
-            mapping = {
-                JobType.FULL_TIME: "F",
-                JobType.PART_TIME: "P",
-                JobType.INTERNSHIP: "I",
-                JobType.CONTRACT: "C",
-                JobType.TEMPORARY: "T",
-            }
-
-            return mapping.get(job_type_enum, "")
-
-        while len(job_list) < scraper_input.results_wanted and page < 1000:
+        seen_ids = set()
+        start = scraper_input.offset // 10 * 10 if scraper_input.offset else 0
+        request_count = 0
+        seconds_old = (
+            scraper_input.hours_old * 3600 if scraper_input.hours_old else None
+        )
+        continue_search = (
+            lambda: len(job_list) < scraper_input.results_wanted and start < 1000
+        )
+        while continue_search():
+            request_count += 1
+            logger.info(
+                f"search page: {request_count} / {math.ceil(scraper_input.results_wanted / 10)}"
+            )
            params = {
                "keywords": scraper_input.search_term,
                "location": scraper_input.location,
                "distance": scraper_input.distance,
                "f_WT": 2 if scraper_input.is_remote else None,
-                "f_JT": job_type_code(scraper_input.job_type)
+                "f_JT": (
+                    self.job_type_code(scraper_input.job_type)
                    if scraper_input.job_type
-                else None,
+                    else None
+                ),
                "pageNum": 0,
-                page: page + scraper_input.offset,
+                "start": start,
                "f_AL": "true" if scraper_input.easy_apply else None,
+                "f_C": (
+                    ",".join(map(str, scraper_input.linkedin_company_ids))
+                    if scraper_input.linkedin_company_ids
+                    else None
+                ),
            }
+            if seconds_old is not None:
+                params["f_TPR"] = f"r{seconds_old}"

            params = {k: v for k, v in params.items() if v is not None}
-
-            params = {k: v for k, v in params.items() if v is not None}
-            retries = 0
-            while retries < self.MAX_RETRIES:
            try:
-                    response = requests.get(
-                        f"{self.url}/jobs-guest/jobs/api/seeMoreJobPostings/search?",
+                response = self.session.get(
+                    f"{self.base_url}/jobs-guest/jobs/api/seeMoreJobPostings/search?",
                    params=params,
-                        allow_redirects=True,
-                        proxies=self.proxy,
                    timeout=10,
                )
-                    response.raise_for_status()
-
-                    break
-                except requests.HTTPError as e:
-                    if hasattr(e, "response") and e.response is not None:
-                        if e.response.status_code == 429:
-                            time.sleep(self.DELAY)
-                            retries += 1
-                            continue
-                        else:
-                            raise LinkedInException(
-                                f"bad response status code: {e.response.status_code}"
+                if response.status_code not in range(200, 400):
+                    if response.status_code == 429:
+                        err = (
+                            f"429 Response - Blocked by LinkedIn for too many requests"
                        )
                    else:
-                        raise
-                except ProxyError as e:
-                    raise LinkedInException("bad proxy")
+                        err = f"LinkedIn response status code {response.status_code}"
+                        err += f" - {response.text}"
+                    logger.error(err)
+                    return JobResponse(jobs=job_list)
            except Exception as e:
-                    raise LinkedInException(str(e))
+                if "Proxy responded with" in str(e):
+                    logger.error(f"LinkedIn: Bad proxy")
                else:
-                # Raise an exception if the maximum number of retries is reached
-                raise LinkedInException(
-                    "Max retries reached, failed to get a valid response"
-                )
+                    logger.error(f"LinkedIn: {str(e)}")
+                return JobResponse(jobs=job_list)

            soup = BeautifulSoup(response.text, "html.parser")
+            job_cards = soup.find_all("div", class_="base-search-card")
+            if len(job_cards) == 0:
+                return JobResponse(jobs=job_list)

-            with ThreadPoolExecutor(max_workers=5) as executor:
-                futures = []
-                for job_card in soup.find_all("div", class_="base-search-card"):
-                    job_url = None
+            for job_card in job_cards:
                href_tag = job_card.find("a", class_="base-card__full-link")
                if href_tag and "href" in href_tag.attrs:
                    href = href_tag.attrs["href"].split("?")[0]
                    job_id = href.split("-")[-1]
-                        job_url = f"{self.url}/jobs/view/{job_id}"

-                    with url_lock:
-                        if job_url in seen_urls:
+                    if job_id in seen_ids:
                        continue
-                        seen_urls.add(job_url)
+                    seen_ids.add(job_id)

-                    futures.append(executor.submit(self.process_job, job_card, job_url))
-
-                for future in as_completed(futures):
                    try:
-                        job_post = future.result()
+                        fetch_desc = scraper_input.linkedin_fetch_description
+                        job_post = self._process_job(job_card, job_id, fetch_desc)
                        if job_post:
                            job_list.append(job_post)
+                        if not continue_search():
+                            break
                    except Exception as e:
-                        raise LinkedInException(
-                            "Exception occurred while processing jobs"
-                        )
-            page += 25
+                        raise LinkedInException(str(e))
+
+            if continue_search():
+                time.sleep(random.uniform(self.delay, self.delay + self.band_delay))
+                start += len(job_list)

        job_list = job_list[: scraper_input.results_wanted]
        return JobResponse(jobs=job_list)

-    def process_job(self, job_card: Tag, job_url: str) -> Optional[JobPost]:
+    def _process_job(
+        self, job_card: Tag, job_id: str, full_descr: bool
+    ) -> Optional[JobPost]:
+        salary_tag = job_card.find("span", class_="job-search-card__salary-info")
+
+        compensation = None
+        if salary_tag:
+            salary_text = salary_tag.get_text(separator=" ").strip()
+            salary_values = [currency_parser(value) for value in salary_text.split("-")]
+            salary_min = salary_values[0]
+            salary_max = salary_values[1]
+            currency = salary_text[0] if salary_text[0] != "$" else "USD"
+
+            compensation = Compensation(
+                min_amount=int(salary_min),
+                max_amount=int(salary_max),
+                currency=currency,
+            )
+
        title_tag = job_card.find("span", class_="sr-only")
        title = title_tag.get_text(strip=True) if title_tag else "N/A"

        company_tag = job_card.find("h4", class_="base-search-card__subtitle")
        company_a_tag = company_tag.find("a") if company_tag else None
+        company_url = (
+            urlunparse(urlparse(company_a_tag.get("href"))._replace(query=""))
+            if company_a_tag and company_a_tag.has_attr("href")
+            else ""
+        )
        company = company_a_tag.get_text(strip=True) if company_a_tag else "N/A"

        metadata_card = job_card.find("div", class_="base-search-card__metadata")
-        location = self.get_location(metadata_card)
+        location = self._get_location(metadata_card)

        datetime_tag = (
            metadata_card.find("time", class_="job-search-card__listdate")
@@ -168,93 +211,92 @@ class LinkedInScraper(Scraper):
            datetime_str = datetime_tag["datetime"]
            try:
                date_posted = datetime.strptime(datetime_str, "%Y-%m-%d")
-            except Exception as e:
+            except:
                date_posted = None
-        benefits_tag = job_card.find("span", class_="result-benefits__text")
-        benefits = " ".join(benefits_tag.get_text().split()) if benefits_tag else None
-
-        description, job_type = self.get_job_description(job_url)
+        job_details = {}
+        if full_descr:
+            job_details = self._get_job_details(job_id)

        return JobPost(
+            id=f"li-{job_id}",
            title=title,
-            description=description,
            company_name=company,
+            company_url=company_url,
            location=location,
            date_posted=date_posted,
-            job_url=job_url,
-            # job_type=[JobType.FULL_TIME],
-            job_type=job_type,
-            benefits=benefits,
-            emails=extract_emails_from_text(description) if description else None,
-            num_urgent_words=count_urgent_words(description) if description else None,
+            job_url=f"{self.base_url}/jobs/view/{job_id}",
+            compensation=compensation,
+            job_type=job_details.get("job_type"),
+            job_level=job_details.get("job_level", "").lower(),
+            company_industry=job_details.get("company_industry"),
+            description=job_details.get("description"),
+            job_url_direct=job_details.get("job_url_direct"),
+            emails=extract_emails_from_text(job_details.get("description")),
+            logo_photo_url=job_details.get("logo_photo_url"),
+            job_function=job_details.get("job_function"),
        )

-    def get_job_description(
-        self, job_page_url: str
-    ) -> tuple[None, None] | tuple[str | None, tuple[str | None, JobType | None]]:
+    def _get_job_details(self, job_id: str) -> dict:
        """
-        Retrieves job description by going to the job page url
+        Retrieves job description and other job details by going to the job page url
        :param job_page_url:
-        :return: description or None
+        :return: dict
        """
        try:
-            response = requests.get(job_page_url, timeout=5, proxies=self.proxy)
+            response = self.session.get(
+                f"{self.base_url}/jobs/view/{job_id}", timeout=5
+            )
            response.raise_for_status()
-        except Exception as e:
-            return None, None
+        except:
+            return {}
+        if "linkedin.com/signup" in response.url:
+            return {}

        soup = BeautifulSoup(response.text, "html.parser")
        div_content = soup.find(
            "div", class_=lambda x: x and "show-more-less-html__markup" in x
        )
-
        description = None
-        if div_content:
-            description = " ".join(div_content.get_text().split()).strip()
+        if div_content is not None:
+            div_content = remove_attributes(div_content)
+            description = div_content.prettify(formatter="html")
+            if self.scraper_input.description_format == DescriptionFormat.MARKDOWN:
+                description = markdown_converter(description)

-        def get_job_type(
-            soup_job_type: BeautifulSoup,
-        ) -> list[JobType] | None:
-            """
-            Gets the job type from job page
-            :param soup_job_type:
-            :return: JobType
-            """
-            h3_tag = soup_job_type.find(
-                "h3",
-                class_="description__job-criteria-subheader",
-                string=lambda text: "Employment type" in text,
+        h3_tag = soup.find(
+            "h3", text=lambda text: text and "Job function" in text.strip()
        )

-            employment_type = None
+        job_function = None
        if h3_tag:
-                employment_type_span = h3_tag.find_next_sibling(
-                    "span",
-                    class_="description__job-criteria-text description__job-criteria-text--criteria",
+            job_function_span = h3_tag.find_next(
+                "span", class_="description__job-criteria-text"
            )
-                if employment_type_span:
-                    employment_type = employment_type_span.get_text(strip=True)
-                    employment_type = employment_type.lower()
-                    employment_type = employment_type.replace("-", "")
+            if job_function_span:
+                job_function = job_function_span.text.strip()

-            return LinkedInScraper.get_enum_from_value(employment_type)
+        logo_photo_url = (
+            logo_image.get("data-delayed-url")
+            if (logo_image := soup.find("img", {"class": "artdeco-entity-image"}))
+            else None
+        )
+        return {
+            "description": description,
+            "job_level": self._parse_job_level(soup),
+            "company_industry": self._parse_company_industry(soup),
+            "job_type": self._parse_job_type(soup),
+            "job_url_direct": self._parse_job_url_direct(soup),
+            "logo_photo_url": logo_photo_url,
+            "job_function": job_function,
+        }

-        return description, get_job_type(soup)
-
-    @staticmethod
-    def get_enum_from_value(value_str):
-        for job_type in JobType:
-            if value_str in job_type.value:
-                return [job_type]
-        return None
-
-    def get_location(self, metadata_card: Optional[Tag]) -> Location:
+    def _get_location(self, metadata_card: Optional[Tag]) -> Location:
        """
        Extracts the location data from the job metadata card.
        :param metadata_card
        :return: location
        """
-        location = Location(country=self.country)
+        location = Location(country=Country.from_string(self.country))
        if metadata_card is not None:
            location_tag = metadata_card.find(
                "span", class_="job-search-card__location"
@@ -266,7 +308,108 @@ class LinkedInScraper(Scraper):
                location = Location(
                    city=city,
                    state=state,
-                    country=self.country,
+                    country=Country.from_string(self.country),
                )
-
+            elif len(parts) == 3:
+                city, state, country = parts
+                country = Country.from_string(country)
+                location = Location(city=city, state=state, country=country)
        return location
+
+    @staticmethod
+    def _parse_job_type(soup_job_type: BeautifulSoup) -> list[JobType] | None:
+        """
+        Gets the job type from job page
+        :param soup_job_type:
+        :return: JobType
+        """
+        h3_tag = soup_job_type.find(
+            "h3",
+            class_="description__job-criteria-subheader",
+            string=lambda text: "Employment type" in text,
+        )
+        employment_type = None
+        if h3_tag:
+            employment_type_span = h3_tag.find_next_sibling(
+                "span",
+                class_="description__job-criteria-text description__job-criteria-text--criteria",
+            )
+            if employment_type_span:
+                employment_type = employment_type_span.get_text(strip=True)
+                employment_type = employment_type.lower()
+                employment_type = employment_type.replace("-", "")
+
+        return [get_enum_from_job_type(employment_type)] if employment_type else []
+
+    @staticmethod
+    def _parse_job_level(soup_job_level: BeautifulSoup) -> str | None:
+        """
+        Gets the job level from job page
+        :param soup_job_level:
+        :return: str
+        """
+        h3_tag = soup_job_level.find(
+            "h3",
+            class_="description__job-criteria-subheader",
+            string=lambda text: "Seniority level" in text,
+        )
+        job_level = None
+        if h3_tag:
+            job_level_span = h3_tag.find_next_sibling(
+                "span",
+                class_="description__job-criteria-text description__job-criteria-text--criteria",
+            )
+            if job_level_span:
+                job_level = job_level_span.get_text(strip=True)
+
+        return job_level
+
+    @staticmethod
+    def _parse_company_industry(soup_industry: BeautifulSoup) -> str | None:
+        """
+        Gets the company industry from job page
+        :param soup_industry:
+        :return: str
+        """
+        h3_tag = soup_industry.find(
+            "h3",
+            class_="description__job-criteria-subheader",
+            string=lambda text: "Industries" in text,
+        )
+        industry = None
+        if h3_tag:
+            industry_span = h3_tag.find_next_sibling(
+                "span",
+                class_="description__job-criteria-text description__job-criteria-text--criteria",
+            )
+            if industry_span:
+                industry = industry_span.get_text(strip=True)
+
+        return industry
+
+    def _parse_job_url_direct(self, soup: BeautifulSoup) -> str | None:
+        """
+        Gets the job url direct from job page
+        :param soup:
+        :return: str
+        """
+        job_url_direct = None
+        job_url_direct_content = soup.find("code", id="applyUrl")
+        if job_url_direct_content:
+            job_url_direct_match = self.job_url_direct_regex.search(
+                job_url_direct_content.decode_contents().strip()
+            )
+            if job_url_direct_match:
+                job_url_direct = unquote(job_url_direct_match.group())
+
+        return job_url_direct
+
+    @staticmethod
+    def job_type_code(job_type_enum: JobType) -> str:
+        return {
+            JobType.FULL_TIME: "F",
+            JobType.PART_TIME: "P",
+            JobType.INTERNSHIP: "I",
+            JobType.CONTRACT: "C",
+            JobType.TEMPORARY: "T",
+        }.get(job_type_enum, "")
--- a/src/jobspy/scrapers/linkedin/constants.py
+++ b/src/jobspy/scrapers/linkedin/constants.py
@@ -0,0 +1,8 @@
+headers = {
+    "authority": "www.linkedin.com",
+    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
+    "accept-language": "en-US,en;q=0.9",
+    "cache-control": "max-age=0",
+    "upgrade-insecure-requests": "1",
+    "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36",
+}
--- a/src/jobspy/scrapers/utils.py
+++ b/src/jobspy/scrapers/utils.py
@@ -1,19 +1,158 @@
+from __future__ import annotations
+
 import re
+import logging
+from itertools import cycle
+
+import requests
 import tls_client
+import numpy as np
+from markdownify import markdownify as md
+from requests.adapters import HTTPAdapter, Retry
+
+from ..jobs import CompensationInterval, JobType


-def count_urgent_words(description: str) -> int:
-    """
-    Count the number of urgent words or phrases in a job description.
-    """
-    urgent_patterns = re.compile(
-        r"\burgen(t|cy)|\bimmediate(ly)?\b|start asap|\bhiring (now|immediate(ly)?)\b",
-        re.IGNORECASE,
+def create_logger(name: str):
+    logger = logging.getLogger(f"JobSpy:{name}")
+    logger.propagate = False
+    if not logger.handlers:
+        logger.setLevel(logging.INFO)
+        console_handler = logging.StreamHandler()
+        format = "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
+        formatter = logging.Formatter(format)
+        console_handler.setFormatter(formatter)
+        logger.addHandler(console_handler)
+    return logger
+
+
+class RotatingProxySession:
+    def __init__(self, proxies=None):
+        if isinstance(proxies, str):
+            self.proxy_cycle = cycle([self.format_proxy(proxies)])
+        elif isinstance(proxies, list):
+            self.proxy_cycle = (
+                cycle([self.format_proxy(proxy) for proxy in proxies])
+                if proxies
+                else None
            )
-    matches = re.findall(urgent_patterns, description)
-    count = len(matches)
+        else:
+            self.proxy_cycle = None

-    return count
+    @staticmethod
+    def format_proxy(proxy):
+        """Utility method to format a proxy string into a dictionary."""
+        if proxy.startswith("http://") or proxy.startswith("https://"):
+            return {"http": proxy, "https": proxy}
+        return {"http": f"http://{proxy}", "https": f"http://{proxy}"}
+
+
+class RequestsRotating(RotatingProxySession, requests.Session):
+
+    def __init__(self, proxies=None, has_retry=False, delay=1, clear_cookies=False):
+        RotatingProxySession.__init__(self, proxies=proxies)
+        requests.Session.__init__(self)
+        self.clear_cookies = clear_cookies
+        self.allow_redirects = True
+        self.setup_session(has_retry, delay)
+
+    def setup_session(self, has_retry, delay):
+        if has_retry:
+            retries = Retry(
+                total=3,
+                connect=3,
+                status=3,
+                status_forcelist=[500, 502, 503, 504, 429],
+                backoff_factor=delay,
+            )
+            adapter = HTTPAdapter(max_retries=retries)
+            self.mount("http://", adapter)
+            self.mount("https://", adapter)
+
+    def request(self, method, url, **kwargs):
+        if self.clear_cookies:
+            self.cookies.clear()
+
+        if self.proxy_cycle:
+            next_proxy = next(self.proxy_cycle)
+            if next_proxy["http"] != "http://localhost":
+                self.proxies = next_proxy
+            else:
+                self.proxies = {}
+        return requests.Session.request(self, method, url, **kwargs)
+
+
+class TLSRotating(RotatingProxySession, tls_client.Session):
+
+    def __init__(self, proxies=None):
+        RotatingProxySession.__init__(self, proxies=proxies)
+        tls_client.Session.__init__(self, random_tls_extension_order=True)
+
+    def execute_request(self, *args, **kwargs):
+        if self.proxy_cycle:
+            next_proxy = next(self.proxy_cycle)
+            if next_proxy["http"] != "http://localhost":
+                self.proxies = next_proxy
+            else:
+                self.proxies = {}
+        response = tls_client.Session.execute_request(self, *args, **kwargs)
+        response.ok = response.status_code in range(200, 400)
+        return response
+
+
+def create_session(
+    *,
+    proxies: dict | str | None = None,
+    ca_cert: str | None = None,
+    is_tls: bool = True,
+    has_retry: bool = False,
+    delay: int = 1,
+    clear_cookies: bool = False,
+) -> requests.Session:
+    """
+    Creates a requests session with optional tls, proxy, and retry settings.
+    :return: A session object
+    """
+    if is_tls:
+        session = TLSRotating(proxies=proxies)
+    else:
+        session = RequestsRotating(
+            proxies=proxies,
+            has_retry=has_retry,
+            delay=delay,
+            clear_cookies=clear_cookies,
+        )
+
+    if ca_cert:
+        session.verify = ca_cert
+
+    return session
+
+
+def set_logger_level(verbose: int = 2):
+    """
+    Adjusts the logger's level. This function allows the logging level to be changed at runtime.
+
+    Parameters:
+    - verbose: int {0, 1, 2} (default=2, all logs)
+    """
+    if verbose is None:
+        return
+    level_name = {2: "INFO", 1: "WARNING", 0: "ERROR"}.get(verbose, "INFO")
+    level = getattr(logging, level_name.upper(), None)
+    if level is not None:
+        for logger_name in logging.root.manager.loggerDict:
+            if logger_name.startswith("JobSpy:"):
+                logging.getLogger(logger_name).setLevel(level)
+    else:
+        raise ValueError(f"Invalid log level: {level_name}")
+
+
+def markdown_converter(description_html: str):
+    if description_html is None:
+        return None
+    markdown = md(description_html)
+    return markdown.strip()


 def extract_emails_from_text(text: str) -> list[str] | None:
@@ -23,22 +162,105 @@ def extract_emails_from_text(text: str) -> list[str] | None:
    return email_regex.findall(text)


-def create_session(proxy: str | None = None):
+def get_enum_from_job_type(job_type_str: str) -> JobType | None:
    """
-    Creates a tls client session
-
-    :return: A session object with or without proxies.
+    Given a string, returns the corresponding JobType enum member if a match is found.
    """
-    session = tls_client.Session(
-        client_identifier="chrome112",
-        random_tls_extension_order=True,
-    )
-    session.proxies = proxy
-    # TODO multiple proxies
-    # if self.proxies:
-    #     session.proxies = {
-    #         "http": random.choice(self.proxies),
-    #         "https": random.choice(self.proxies),
-    #     }
+    res = None
+    for job_type in JobType:
+        if job_type_str in job_type.value:
+            res = job_type
+    return res

-    return session
+
+def currency_parser(cur_str):
+    # Remove any non-numerical characters
+    # except for ',' '.' or '-' (e.g. EUR)
+    cur_str = re.sub("[^-0-9.,]", "", cur_str)
+    # Remove any 000s separators (either , or .)
+    cur_str = re.sub("[.,]", "", cur_str[:-3]) + cur_str[-3:]
+
+    if "." in list(cur_str[-3:]):
+        num = float(cur_str)
+    elif "," in list(cur_str[-3:]):
+        num = float(cur_str.replace(",", "."))
+    else:
+        num = float(cur_str)
+
+    return np.round(num, 2)
+
+
+def remove_attributes(tag):
+    for attr in list(tag.attrs):
+        del tag[attr]
+    return tag
+
+
+def extract_salary(
+    salary_str,
+    lower_limit=1000,
+    upper_limit=700000,
+    hourly_threshold=350,
+    monthly_threshold=30000,
+    enforce_annual_salary=False,
+):
+    """
+    Extracts salary information from a string and returns the salary interval, min and max salary values, and currency.
+    (TODO: Needs test cases as the regex is complicated and may not cover all edge cases)
+    """
+    if not salary_str:
+        return None, None, None, None
+
+    annual_max_salary = None
+    min_max_pattern = r"\$(\d+(?:,\d+)?(?:\.\d+)?)([kK]?)\s*[-—–]\s*(?:\$)?(\d+(?:,\d+)?(?:\.\d+)?)([kK]?)"
+
+    def to_int(s):
+        return int(float(s.replace(",", "")))
+
+    def convert_hourly_to_annual(hourly_wage):
+        return hourly_wage * 2080
+
+    def convert_monthly_to_annual(monthly_wage):
+        return monthly_wage * 12
+
+    match = re.search(min_max_pattern, salary_str)
+
+    if match:
+        min_salary = to_int(match.group(1))
+        max_salary = to_int(match.group(3))
+        # Handle 'k' suffix for min and max salaries independently
+        if "k" in match.group(2).lower() or "k" in match.group(4).lower():
+            min_salary *= 1000
+            max_salary *= 1000
+
+        # Convert to annual if less than the hourly threshold
+        if min_salary < hourly_threshold:
+            interval = CompensationInterval.HOURLY.value
+            annual_min_salary = convert_hourly_to_annual(min_salary)
+            if max_salary < hourly_threshold:
+                annual_max_salary = convert_hourly_to_annual(max_salary)
+
+        elif min_salary < monthly_threshold:
+            interval = CompensationInterval.MONTHLY.value
+            annual_min_salary = convert_monthly_to_annual(min_salary)
+            if max_salary < monthly_threshold:
+                annual_max_salary = convert_monthly_to_annual(max_salary)
+
+        else:
+            interval = CompensationInterval.YEARLY.value
+            annual_min_salary = min_salary
+            annual_max_salary = max_salary
+
+        # Ensure salary range is within specified limits
+        if not annual_max_salary:
+            return None, None, None, None
+        if (
+            lower_limit <= annual_min_salary <= upper_limit
+            and lower_limit <= annual_max_salary <= upper_limit
+            and annual_min_salary < annual_max_salary
+        ):
+            if enforce_annual_salary:
+                return interval, annual_min_salary, annual_max_salary, "USD"
+            else:
+                return interval, min_salary, max_salary, "USD"
+    return None, None, None, None
--- a/src/jobspy/scrapers/ziprecruiter/init.py
+++ b/src/jobspy/scrapers/ziprecruiter/init.py
@@ -4,472 +4,244 @@ jobspy.scrapers.ziprecruiter

 This module contains routines to scrape ZipRecruiter.
 """
-import math
+
+from __future__ import annotations
+
 import json
+import math
 import re
-from datetime import datetime, date
+import time
+from datetime import datetime
 from typing import Optional, Tuple, Any
-from urllib.parse import urlparse, parse_qs, urlunparse

-import requests
+from concurrent.futures import ThreadPoolExecutor
+
 from bs4 import BeautifulSoup
-from bs4.element import Tag
-from concurrent.futures import ThreadPoolExecutor, Future

+from .constants import headers
 from .. import Scraper, ScraperInput, Site
-from ..exceptions import ZipRecruiterException
-from ..utils import count_urgent_words, extract_emails_from_text, create_session
+from ..utils import (
+    extract_emails_from_text,
+    create_session,
+    markdown_converter,
+    remove_attributes,
+    create_logger,
+)
 from ...jobs import (
    JobPost,
    Compensation,
-    CompensationInterval,
    Location,
    JobResponse,
    JobType,
    Country,
+    DescriptionFormat,
 )

+logger = create_logger("ZipRecruiter")
+

 class ZipRecruiterScraper(Scraper):
-    def __init__(self, proxy: Optional[str] = None):
-        """
-        Initializes LinkedInScraper with the ZipRecruiter job search url
-        """
-        site = Site(Site.ZIP_RECRUITER)
-        self.url = "https://www.ziprecruiter.com"
-        super().__init__(site, proxy=proxy)
+    base_url = "https://www.ziprecruiter.com"
+    api_url = "https://api.ziprecruiter.com"

+    def __init__(
+        self, proxies: list[str] | str | None = None, ca_cert: str | None = None
+    ):
+        """
+        Initializes ZipRecruiterScraper with the ZipRecruiter job search url
+        """
+        super().__init__(Site.ZIP_RECRUITER, proxies=proxies)
+
+        self.scraper_input = None
+        self.session = create_session(proxies=proxies, ca_cert=ca_cert)
+        self.session.headers.update(headers)
+        self._get_cookies()
+
+        self.delay = 5
        self.jobs_per_page = 20
        self.seen_urls = set()

-    def find_jobs_in_page(
-        self, scraper_input: ScraperInput, page: int
-    ) -> list[JobPost]:
+    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
+        """
+        Scrapes ZipRecruiter for jobs with scraper_input criteria.
+        :param scraper_input: Information about job search criteria.
+        :return: JobResponse containing a list of jobs.
+        """
+        self.scraper_input = scraper_input
+        job_list: list[JobPost] = []
+        continue_token = None
+
+        max_pages = math.ceil(scraper_input.results_wanted / self.jobs_per_page)
+        for page in range(1, max_pages + 1):
+            if len(job_list) >= scraper_input.results_wanted:
+                break
+            if page > 1:
+                time.sleep(self.delay)
+            logger.info(f"search page: {page} / {max_pages}")
+            jobs_on_page, continue_token = self._find_jobs_in_page(
+                scraper_input, continue_token
+            )
+            if jobs_on_page:
+                job_list.extend(jobs_on_page)
+            else:
+                break
+            if not continue_token:
+                break
+        return JobResponse(jobs=job_list[: scraper_input.results_wanted])
+
+    def _find_jobs_in_page(
+        self, scraper_input: ScraperInput, continue_token: str | None = None
+    ) -> Tuple[list[JobPost], Optional[str]]:
        """
        Scrapes a page of ZipRecruiter for jobs with scraper_input criteria
        :param scraper_input:
-        :param page:
+        :param continue_token:
        :return: jobs found on page
        """
-        session = create_session(self.proxy)
+        jobs_list = []
+        params = self._add_params(scraper_input)
+        if continue_token:
+            params["continue_from"] = continue_token
        try:
-            response = session.get(
-                f"{self.url}/jobs-search",
-                headers=self.headers(),
-                params=self.add_params(scraper_input, page),
-                allow_redirects=True,
-                timeout_seconds=10,
-            )
-            if response.status_code != 200:
-                raise ZipRecruiterException(
-                    f"bad response status code: {response.status_code}"
-                )
+            res = self.session.get(f"{self.api_url}/jobs-app/jobs", params=params)
+            if res.status_code not in range(200, 400):
+                if res.status_code == 429:
+                    err = "429 Response - Blocked by ZipRecruiter for too many requests"
+                else:
+                    err = f"ZipRecruiter response status code {res.status_code}"
+                    err += f" with response: {res.text}"  # ZipRecruiter likely not available in EU
+                logger.error(err)
+                return jobs_list, ""
        except Exception as e:
-            if "Proxy responded with non 200 code" in str(e):
-                raise ZipRecruiterException("bad proxy")
-            raise ZipRecruiterException(str(e))
+            if "Proxy responded with" in str(e):
+                logger.error(f"Indeed: Bad proxy")
            else:
-            soup = BeautifulSoup(response.text, "html.parser")
-            js_tag = soup.find("script", {"id": "js_variables"})
+                logger.error(f"Indeed: {str(e)}")
+            return jobs_list, ""

-        if js_tag:
-            page_json = json.loads(js_tag.string)
-            jobs_list = page_json.get("jobList")
-            if jobs_list:
-                page_variant = "javascript"
-                # print('type javascript', len(jobs_list))
-            else:
-                page_variant = "html_2"
-                jobs_list = soup.find_all("div", {"class": "job_content"})
-                # print('type 2 html', len(jobs_list))
-        else:
-            page_variant = "html_1"
-            jobs_list = soup.find_all("li", {"class": "job-listing"})
-            # print('type 1 html', len(jobs_list))
+        res_data = res.json()
+        jobs_list = res_data.get("jobs", [])
+        next_continue_token = res_data.get("continue", None)
+        with ThreadPoolExecutor(max_workers=self.jobs_per_page) as executor:
+            job_results = [executor.submit(self._process_job, job) for job in jobs_list]

-        with ThreadPoolExecutor(max_workers=10) as executor:
-            if page_variant == "javascript":
-                job_results = [
-                    executor.submit(self.process_job_javascript, job)
-                    for job in jobs_list
-                ]
-            elif page_variant == "html_1":
-                job_results = [
-                    executor.submit(self.process_job_html_1, job) for job in jobs_list
-                ]
-            elif page_variant == "html_2":
-                job_results = [
-                    executor.submit(self.process_job_html_2, job) for job in jobs_list
-                ]
+        job_list = list(filter(None, (result.result() for result in job_results)))
+        return job_list, next_continue_token

-        job_list = [result.result() for result in job_results if result.result()]
-        return job_list
-
-    def scrape(self, scraper_input: ScraperInput) -> JobResponse:
+    def _process_job(self, job: dict) -> JobPost | None:
        """
-        Scrapes ZipRecruiter for jobs with scraper_input criteria
-        :param scraper_input:
-        :return: job_response
+        Processes an individual job dict from the response
        """
-        start_page = (
-            (scraper_input.offset // self.jobs_per_page) + 1
-            if scraper_input.offset
-            else 1
-        )
-        #: get first page to initialize session
-        job_list: list[JobPost] = self.find_jobs_in_page(scraper_input, start_page)
-        pages_to_process = max(
-            3, math.ceil(scraper_input.results_wanted / self.jobs_per_page)
+        title = job.get("name")
+        job_url = f"{self.base_url}/jobs//j?lvk={job['listing_key']}"
+        if job_url in self.seen_urls:
+            return
+        self.seen_urls.add(job_url)
+
+        description = job.get("job_description", "").strip()
+        listing_type = job.get("buyer_type", "")
+        description = (
+            markdown_converter(description)
+            if self.scraper_input.description_format == DescriptionFormat.MARKDOWN
+            else description
        )
+        company = job.get("hiring_company", {}).get("name")
+        country_value = "usa" if job.get("job_country") == "US" else "canada"
+        country_enum = Country.from_string(country_value)

-        with ThreadPoolExecutor(max_workers=10) as executor:
-            futures: list[Future] = [
-                executor.submit(self.find_jobs_in_page, scraper_input, page)
-                for page in range(start_page + 1, start_page + pages_to_process + 2)
-            ]
-
-            for future in futures:
-                jobs = future.result()
-
-                job_list += jobs
-
-        job_list = job_list[: scraper_input.results_wanted]
-        return JobResponse(jobs=job_list)
-
-    def process_job_javascript(self, job: dict) -> JobPost:
-        """the most common type of jobs page on ZR"""
-        title = job.get("Title")
-        job_url = job.get("JobURL")
-
-        description, updated_job_url = self.get_description(job_url)
-        # job_url = updated_job_url if updated_job_url else job_url
-        if description is None:
-            description = BeautifulSoup(
-                job.get("Snippet", "").strip(), "html.parser"
-            ).get_text()
-
-        company = job.get("OrgName")
        location = Location(
-            city=job.get("City"), state=job.get("State"), country=Country.US_CANADA
+            city=job.get("job_city"), state=job.get("job_state"), country=country_enum
        )
-        job_type = ZipRecruiterScraper.get_job_type_enum(
-            job.get("EmploymentType", "").replace("-", "").lower()
+        job_type = self._get_job_type_enum(
+            job.get("employment_type", "").replace("_", "").lower()
        )
-
-        formatted_salary = job.get("FormattedSalaryShort", "")
-        salary_parts = formatted_salary.split(" ")
-
-        min_salary_str = salary_parts[0][1:].replace(",", "")
-        if "." in min_salary_str:
-            min_amount = int(float(min_salary_str) * 1000)
-        else:
-            min_amount = int(min_salary_str.replace("K", "000"))
-
-        if len(salary_parts) >= 3 and salary_parts[2].startswith("$"):
-            max_salary_str = salary_parts[2][1:].replace(",", "")
-            if "." in max_salary_str:
-                max_amount = int(float(max_salary_str) * 1000)
-            else:
-                max_amount = int(max_salary_str.replace("K", "000"))
-        else:
-            max_amount = 0
-
-        compensation = Compensation(
-            interval=CompensationInterval.YEARLY,
-            min_amount=min_amount,
-            max_amount=max_amount,
-            currency="USD/CAD",
-        )
-        save_job_url = job.get("SaveJobURL", "")
-        posted_time_match = re.search(
-            r"posted_time=(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}Z)", save_job_url
-        )
-        if posted_time_match:
-            date_time_str = posted_time_match.group(1)
-            date_posted_obj = datetime.strptime(date_time_str, "%Y-%m-%dT%H:%M:%SZ")
-            date_posted = date_posted_obj.date()
-        else:
-            date_posted = date.today()
+        date_posted = datetime.fromisoformat(job["posted_time"].rstrip("Z")).date()
+        comp_interval = job.get("compensation_interval")
+        comp_interval = "yearly" if comp_interval == "annual" else comp_interval
+        comp_min = int(job["compensation_min"]) if "compensation_min" in job else None
+        comp_max = int(job["compensation_max"]) if "compensation_max" in job else None
+        comp_currency = job.get("compensation_currency")
+        description_full, job_url_direct = self._get_descr(job_url)

        return JobPost(
+            id=f'zr-{job["listing_key"]}',
            title=title,
            company_name=company,
            location=location,
            job_type=job_type,
-            compensation=compensation,
+            compensation=Compensation(
+                interval=comp_interval,
+                min_amount=comp_min,
+                max_amount=comp_max,
+                currency=comp_currency,
+            ),
            date_posted=date_posted,
            job_url=job_url,
-            description=description,
+            description=description_full if description_full else description,
            emails=extract_emails_from_text(description) if description else None,
-            num_urgent_words=count_urgent_words(description) if description else None,
+            job_url_direct=job_url_direct,
+            listing_type=listing_type,
        )

-    def process_job_html_2(self, job: Tag) -> Optional[JobPost]:
-        """
-        second most common type of jobs page on ZR after process_job_javascript()
-        Parses a job from the job content tag for a second variat of HTML that ZR uses
-        :param job: BeautifulSoup Tag for one job post
-        :return JobPost
-        """
-        job_url = job.find("a", class_="job_link")["href"]
-        title = job.find("h2", class_="title").text
-        company = job.find("a", class_="company_name").text.strip()
-
-        description, updated_job_url = self.get_description(job_url)
-        # job_url = updated_job_url if updated_job_url else job_url
-        if description is None:
-            description = job.find("p", class_="job_snippet").get_text().strip()
-
-        job_type_text = job.find("li", class_="perk_item perk_type")
-        job_type = None
-        if job_type_text:
-            job_type_text = (
-                job_type_text.get_text()
-                .strip()
-                .lower()
-                .replace("-", "")
-                .replace(" ", "")
+    def _get_descr(self, job_url):
+        res = self.session.get(job_url, allow_redirects=True)
+        description_full = job_url_direct = None
+        if res.ok:
+            soup = BeautifulSoup(res.text, "html.parser")
+            job_descr_div = soup.find("div", class_="job_description")
+            company_descr_section = soup.find("section", class_="company_description")
+            job_description_clean = (
+                remove_attributes(job_descr_div).prettify(formatter="html")
+                if job_descr_div
+                else ""
            )
-            job_type = ZipRecruiterScraper.get_job_type_enum(job_type_text)
-        date_posted = ZipRecruiterScraper.get_date_posted(job)
-
-        job_post = JobPost(
-            title=title,
-            company_name=company,
-            location=ZipRecruiterScraper.get_location(job),
-            job_type=job_type,
-            compensation=ZipRecruiterScraper.get_compensation(job),
-            date_posted=date_posted,
-            job_url=job_url,
-            description=description,
-            emails=extract_emails_from_text(description) if description else None,
-            num_urgent_words=count_urgent_words(description) if description else None,
+            company_description_clean = (
+                remove_attributes(company_descr_section).prettify(formatter="html")
+                if company_descr_section
+                else ""
            )
-        return job_post
+            description_full = job_description_clean + company_description_clean
+            script_tag = soup.find("script", type="application/json")
+            if script_tag:
+                job_json = json.loads(script_tag.string)
+                job_url_val = job_json["model"].get("saveJobURL", "")
+                m = re.search(r"job_url=(.+)", job_url_val)
+                if m:
+                    job_url_direct = m.group(1)

-    def process_job_html_1(self, job: Tag) -> Optional[JobPost]:
-        """
-        TODO this method isnt finished due to not encountering this type of html often
-        least common type of jobs page on ZR (rarely found)
-        Parses a job from the job content tag
-        :param job: BeautifulSoup Tag for one job post
-        :return JobPost
-        """
-        job_url = job.find("a", {"class": "job_link"})["href"]
-        # job_url = self.cleanurl(job.find("a", {"class": "job_link"})["href"])
-        if job_url in self.seen_urls:
-            return None
+            if self.scraper_input.description_format == DescriptionFormat.MARKDOWN:
+                description_full = markdown_converter(description_full)

-        title = job.find("h2", {"class": "title"}).text
-        company = job.find("a", {"class": "company_name"}).text.strip()
+        return description_full, job_url_direct

-        description, _ = self.get_description(job_url)
-        # job_url = updated_job_url if updated_job_url else job_url
-        # get description from jobs listing page if get_description from the specific job page fails
-        if description is None:
-            description = job.find("p", {"class": "job_snippet"}).text.strip()
-
-        job_type_element = job.find("li", {"class": "perk_item perk_type"})
-        job_type = None
-        if job_type_element:
-            job_type_text = (
-                job_type_element.text.strip().lower().replace("_", "").replace(" ", "")
-            )
-            job_type = ZipRecruiterScraper.get_job_type_enum(job_type_text)
-
-        date_posted = ZipRecruiterScraper.get_date_posted(job)
-
-        job_post = JobPost(
-            title=title,
-            description=description,
-            company_name=company,
-            location=ZipRecruiterScraper.get_location(job),
-            job_type=job_type,
-            compensation=ZipRecruiterScraper.get_compensation(job),
-            date_posted=date_posted,
-            job_url=job_url,
-            emails=extract_emails_from_text(description),
-            num_urgent_words=count_urgent_words(description),
-        )
-        return job_post
+    def _get_cookies(self):
+        data = "event_type=session&logged_in=false&number_of_retry=1&property=model%3AiPhone&property=os%3AiOS&property=locale%3Aen_us&property=app_build_number%3A4734&property=app_version%3A91.0&property=manufacturer%3AApple&property=timestamp%3A2024-01-12T12%3A04%3A42-06%3A00&property=screen_height%3A852&property=os_version%3A16.6.1&property=source%3Ainstall&property=screen_width%3A393&property=device_model%3AiPhone%2014%20Pro&property=brand%3AApple"
+        url = f"{self.api_url}/jobs-app/event"
+        self.session.post(url, data=data)

    @staticmethod
-    def get_job_type_enum(job_type_str: str) -> list[JobType] | None:
+    def _get_job_type_enum(job_type_str: str) -> list[JobType] | None:
        for job_type in JobType:
            if job_type_str in job_type.value:
                return [job_type]
        return None

-    def get_description(self, job_page_url: str) -> Tuple[str | None, str | None]:
-        """
-        Retrieves job description by going to the job page url
-        :param job_page_url:
-        :return: description or None, response url
-        """
-        try:
-            session = create_session(self.proxy)
-            response = session.get(
-                job_page_url,
-                headers=self.headers(),
-                allow_redirects=True,
-                timeout_seconds=5,
-            )
-            if response.status_code not in range(200, 400):
-                return None, None
-        except Exception as e:
-            return None, None
-
-        html_string = response.content
-        soup_job = BeautifulSoup(html_string, "html.parser")
-
-        job_description_div = soup_job.find("div", {"class": "job_description"})
-        if job_description_div:
-            return job_description_div.text.strip(), response.url
-        return None, response.url
-
    @staticmethod
-    def add_params(scraper_input, page) -> dict[str, str | Any]:
+    def _add_params(scraper_input) -> dict[str, str | Any]:
        params = {
            "search": scraper_input.search_term,
            "location": scraper_input.location,
-            "page": page,
-            "form": "jobs-landing",
        }
-        job_type_value = None
+        if scraper_input.hours_old:
+            params["days"] = max(scraper_input.hours_old // 24, 1)
+        job_type_map = {JobType.FULL_TIME: "full_time", JobType.PART_TIME: "part_time"}
        if scraper_input.job_type:
-            if scraper_input.job_type.value == "fulltime":
-                job_type_value = "full_time"
-            elif scraper_input.job_type.value == "parttime":
-                job_type_value = "part_time"
-            else:
-                job_type_value = scraper_input.job_type.value
-
-        if job_type_value:
-            params[
-                "refine_by_employment"
-            ] = f"employment_type:employment_type:{job_type_value}"
-
+            job_type = scraper_input.job_type
+            params["employment_type"] = job_type_map.get(job_type, job_type.value[0])
+        if scraper_input.easy_apply:
+            params["zipapply"] = 1
        if scraper_input.is_remote:
-            params["refine_by_location_type"] = "only_remote"
-
+            params["remote"] = 1
        if scraper_input.distance:
            params["radius"] = scraper_input.distance
-
-        return params
-
-    @staticmethod
-    def get_interval(interval_str: str):
-        """
-         Maps the interval alias to its appropriate CompensationInterval.
-        :param interval_str
-        :return: CompensationInterval
-        """
-        interval_alias = {"annually": CompensationInterval.YEARLY}
-        interval_str = interval_str.lower()
-
-        if interval_str in interval_alias:
-            return interval_alias[interval_str]
-
-        return CompensationInterval(interval_str)
-
-    @staticmethod
-    def get_date_posted(job: Tag) -> Optional[datetime.date]:
-        """
-        Extracts the date a job was posted
-        :param job
-        :return: date the job was posted or None
-        """
-        button = job.find(
-            "button", {"class": "action_input save_job zrs_btn_secondary_200"}
-        )
-        if not button:
-            return None
-
-        url_time = button.get("data-href", "")
-        url_components = urlparse(url_time)
-        params = parse_qs(url_components.query)
-        posted_time_str = params.get("posted_time", [None])[0]
-
-        if posted_time_str:
-            posted_date = datetime.strptime(
-                posted_time_str, "%Y-%m-%dT%H:%M:%SZ"
-            ).date()
-            return posted_date
-
-        return None
-
-    @staticmethod
-    def get_compensation(job: Tag) -> Optional[Compensation]:
-        """
-        Parses the compensation tag from the job BeautifulSoup object
-        :param job
-        :return: Compensation object or None
-        """
-        pay_element = job.find("li", {"class": "perk_item perk_pay"})
-        if pay_element is None:
-            return None
-        pay = pay_element.find("div", {"class": "value"}).find("span").text.strip()
-
-        def create_compensation_object(pay_string: str) -> Compensation:
-            """
-            Creates a Compensation object from a pay_string
-            :param pay_string
-            :return: compensation
-            """
-            interval = ZipRecruiterScraper.get_interval(pay_string.split()[-1])
-
-            amounts = []
-            for amount in pay_string.split("to"):
-                amount = amount.replace(",", "").strip("$ ").split(" ")[0]
-                if "K" in amount:
-                    amount = amount.replace("K", "")
-                    amount = int(float(amount)) * 1000
-                else:
-                    amount = int(float(amount))
-                amounts.append(amount)
-
-            compensation = Compensation(
-                interval=interval,
-                min_amount=min(amounts),
-                max_amount=max(amounts),
-                currency="USD/CAD",
-            )
-
-            return compensation
-
-        return create_compensation_object(pay)
-
-    @staticmethod
-    def get_location(job: Tag) -> Location:
-        """
-        Extracts the job location from BeatifulSoup object
-        :param job:
-        :return: location
-        """
-        location_link = job.find("a", {"class": "company_location"})
-        if location_link is not None:
-            location_string = location_link.text.strip()
-            parts = location_string.split(", ")
-            if len(parts) == 2:
-                city, state = parts
-            else:
-                city, state = None, None
-        else:
-            city, state = None, None
-        return Location(city=city, state=state, country=Country.US_CANADA)
-
-    @staticmethod
-    def headers() -> dict:
-        """
-        Returns headers needed for requests
-        :return: dict - Dictionary containing headers
-        """
-        return {
-            "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36"
-        }
-
-    # @staticmethod
-    # def cleanurl(url) -> str:
-    #     parsed_url = urlparse(url)
-    #
-    #     return urlunparse((parsed_url.scheme, parsed_url.netloc, parsed_url.path, parsed_url.params, '', ''))
+        return {k: v for k, v in params.items() if v is not None}
--- a/src/jobspy/scrapers/ziprecruiter/constants.py
+++ b/src/jobspy/scrapers/ziprecruiter/constants.py
@@ -0,0 +1,10 @@
+headers = {
+    "Host": "api.ziprecruiter.com",
+    "accept": "*/*",
+    "x-zr-zva-override": "100000000;vid:ZT1huzm_EQlDTVEc",
+    "x-pushnotificationid": "0ff4983d38d7fc5b3370297f2bcffcf4b3321c418f5c22dd152a0264707602a0",
+    "x-deviceid": "D77B3A92-E589-46A4-8A39-6EF6F1D86006",
+    "user-agent": "Job Search/87.0 (iPhone; CPU iOS 16_6_1 like Mac OS X)",
+    "authorization": "Basic YTBlZjMyZDYtN2I0Yy00MWVkLWEyODMtYTI1NDAzMzI0YTcyOg==",
+    "accept-language": "en-US,en;q=0.9",
+}
--- a/src/tests/init.py
+++ b/src/tests/init.py
--- a/src/tests/test_all.py
+++ b/src/tests/test_all.py
@@ -1,14 +0,0 @@
-from ..jobspy import scrape_jobs
-import pandas as pd
-
-
-def test_all():
-    result = scrape_jobs(
-        site_name=["linkedin", "indeed", "zip_recruiter"],
-        search_term="software engineer",
-        results_wanted=5,
-    )
-
-    assert (
-        isinstance(result, pd.DataFrame) and not result.empty
-    ), "Result should be a non-empty DataFrame"
--- a/src/tests/test_linkedin.py
+++ b/src/tests/test_linkedin.py
@@ -1,12 +0,0 @@
-from ..jobspy import scrape_jobs
-import pandas as pd
-
-
-def test_linkedin():
-    result = scrape_jobs(
-        site_name="linkedin",
-        search_term="software engineer",
-    )
-    assert (
-        isinstance(result, pd.DataFrame) and not result.empty
-    ), "Result should be a non-empty DataFrame"
--- a/src/tests/test_ziprecruiter.py
+++ b/src/tests/test_ziprecruiter.py
@@ -1,13 +0,0 @@
-from ..jobspy import scrape_jobs
-import pandas as pd
-
-
-def test_ziprecruiter():
-    result = scrape_jobs(
-        site_name="zip_recruiter",
-        search_term="software engineer",
-    )
-
-    assert (
-        isinstance(result, pd.DataFrame) and not result.empty
-    ), "Result should be a non-empty DataFrame"
--- a/tests/init.py
+++ b/tests/init.py
--- a/tests/test_all.py
+++ b/tests/test_all.py
@@ -0,0 +1,18 @@
+from jobspy import scrape_jobs
+import pandas as pd
+
+
+def test_all():
+    sites = [
+        "indeed",
+        "glassdoor",
+    ]  # ziprecruiter/linkedin needs good ip, and temp fix to pass test on ci
+    result = scrape_jobs(
+        site_name=sites,
+        search_term="engineer",
+        results_wanted=5,
+    )
+
+    assert (
+        isinstance(result, pd.DataFrame) and len(result) == len(sites) * 5
+    ), "Result should be a non-empty DataFrame"
--- a/tests/test_glassdoor.py
+++ b/tests/test_glassdoor.py
@@ -0,0 +1,13 @@
+from jobspy import scrape_jobs
+import pandas as pd
+
+
+def test_glassdoor():
+    result = scrape_jobs(
+        site_name="glassdoor",
+        search_term="engineer",
+        results_wanted=5,
+    )
+    assert (
+        isinstance(result, pd.DataFrame) and len(result) == 5
+    ), "Result should be a non-empty DataFrame"
--- a/src/tests/test_indeed.py
+++ b/src/tests/test_indeed.py
@@ -1,12 +1,13 @@
-from ..jobspy import scrape_jobs
+from jobspy import scrape_jobs
 import pandas as pd


 def test_indeed():
    result = scrape_jobs(
        site_name="indeed",
-        search_term="software engineer",
+        search_term="engineer",
+        results_wanted=5,
    )
    assert (
-        isinstance(result, pd.DataFrame) and not result.empty
+        isinstance(result, pd.DataFrame) and len(result) == 5
    ), "Result should be a non-empty DataFrame"
--- a/tests/test_linkedin.py
+++ b/tests/test_linkedin.py
@@ -0,0 +1,9 @@
+from jobspy import scrape_jobs
+import pandas as pd
+
+
+def test_linkedin():
+    result = scrape_jobs(site_name="linkedin", search_term="engineer", results_wanted=5)
+    assert (
+        isinstance(result, pd.DataFrame) and len(result) == 5
+    ), "Result should be a non-empty DataFrame"
--- a/tests/test_ziprecruiter.py
+++ b/tests/test_ziprecruiter.py
@@ -0,0 +1,12 @@
+from jobspy import scrape_jobs
+import pandas as pd
+
+
+def test_ziprecruiter():
+    result = scrape_jobs(
+        site_name="zip_recruiter", search_term="software engineer", results_wanted=5
+    )
+
+    assert (
+        isinstance(result, pd.DataFrame) and len(result) == 5
+    ), "Result should be a non-empty DataFrame"
Author	SHA1	Message	Date
Cullen Watson	f395597fdd	fix(indeed): offset	2024-10-22 19:25:07 -05:00
Cullen Watson	6372e41bd9	chore:version	2024-10-20 00:19:31 -05:00
Olzhas Arystanov	6c869decb8	build(deps): bump markdownify to 0.13.1 (#211 )	2024-10-20 00:18:44 -05:00
Cullen Watson	9f4083380d	indeed:remove tpe (#210 )	2024-10-19 18:01:59 -05:00
Olzhas Arystanov	9207ab56f6	fix: extract tests out of src (#209 )	2024-10-19 16:56:38 -05:00
Cullen Watson	757a94853e	chore:version	2024-10-08 17:49:06 -05:00
Marcel Gozalbo Baró	6bc191d5c7	FEATURE: Add the "ca_cert" setting for providing a Certification Authority certificate in order to use proxies requiring it. (#204 )	2024-10-08 17:46:46 -05:00
Cullen Watson	0cc34287f7	fix:turkey	2024-10-02 01:31:00 -05:00
Anton Pikhteryev	923979093b	Add Malta for linkedin country support (#198 )	2024-09-19 20:41:22 -05:00
Cullen Watson	286f0e4487	docs:readme	2024-09-18 18:49:41 -05:00
Cullen Watson	f7b29d43a2	fix(indeed):sort relevance not date (#197 )	2024-09-18 18:42:25 -05:00
Cullen Watson	6f1490458c	fix key error (#186 )	2024-08-14 02:54:40 -05:00
Cullen Watson	6bb7d81ba8	change linkedin ep (#185 )	2024-08-14 02:39:43 -05:00
Cullen Watson	0e046432d1	fix:variable bug (#181 )	2024-08-05 12:47:55 -05:00
Cullen Watson	209e0e65b6	fix:malaysia indeed (#180 )	2024-08-03 22:48:53 -05:00
Cullen Watson	8570c0651e	fix:key error (#176 )	2024-07-21 13:05:18 -05:00
Cullen Watson	8678b0bbe4	enh: test on pr (#174 )	2024-07-19 14:25:25 -05:00
Cullen Watson	60d4d911c9	lock file (#173 )	2024-07-17 21:21:22 -05:00
Lluís Salord Quetglas	2a0cba8c7e	FEAT: Optional convertion to annual and know salary source (#170 )	2024-07-17 21:05:33 -05:00
Mason DePalma	de70189fa2	Update pyproject.toml (#172 ) Changed Numpy to the most recent version so the package can properly install	2024-07-17 20:54:08 -05:00
Cullen Watson	b55c0eb86d	docs:readme	2024-07-16 19:24:38 -05:00
Cullen Watson	88c95c4ad5	enh: estimated salary (#169 )	2024-07-16 19:20:34 -05:00
Cullen Watson	d8d33d602f	docs: readme	2024-07-15 21:30:11 -05:00
Cullen Watson	6330c14879	minor fix	2024-07-15 21:19:01 -05:00
Ali Bakhshi Ilani	48631ea271	Add company industry and job level to linkedin scraper (#166 )	2024-07-15 21:07:39 -05:00
Cullen Watson	edffe18e65	enh: listing source (#168 )	2024-07-15 20:30:04 -05:00
Lluís Salord Quetglas	0988230a24	FEAT: Add Glassdoor logo data if available (#167 )	2024-07-15 20:25:18 -05:00
Cullen Watson	d000a81eb3	Salary parse (#163 )	2024-06-09 17:45:38 -05:00
Cullen Watson	ccb0c17660	enh: ziprecruiter full description (#162 )	2024-06-09 16:21:01 -05:00
Cullen Watson	df339610fa	docs: readme	2024-05-29 19:32:32 -05:00
Cullen Watson	c501006bd8	docs: readme	2024-05-28 16:04:26 -05:00
Cullen Watson	89a3ee231c	enh(li): job function (#160 )	2024-05-28 16:01:29 -05:00
Cullen	6439f71433	chore: version	2024-05-28 15:39:24 -05:00
adamagassi	7f6271b2e0	LinkedIn scraper fixes: (#159 ) Correct initial page offset calculation Separate page variable from request counter Fix job offset starting value Increment offset by number of jobs returned instead of expected value	2024-05-28 15:38:13 -05:00
Cullen Watson	5cb7ffe5fd	enh: proxies (#157 ) * enh: proxies * enh: proxies	2024-05-25 14:04:09 -05:00
Cullen Watson	cd29f79796	docs: readme	2024-05-25 11:46:23 -05:00
Cullen Watson	65d2e5e707	Update pyproject.toml	2024-05-20 11:46:36 -05:00
fasih hussain	08d63a87a2	chore: id added for JobPost schema (#152 )	2024-05-20 11:45:52 -05:00
Cullen	1ffdb1756f	fix: dup line	2024-04-30 12:11:48 -05:00
Cullen Watson	1185693422	delete empty file	2024-04-30 12:06:20 -05:00
Lluís Salord Quetglas	dcd7144318	FIX: Allow Indeed search term with complex syntax (#139 )	2024-04-30 12:05:43 -05:00
Cullen Watson	bf73c061bd	enh: linkedin company logo (#141 )	2024-04-30 12:03:10 -05:00
Lluís Salord Quetglas	8dd08ed9fd	FEAT: Allow LinkedIn scraper to get external job apply url (#140 )	2024-04-30 11:36:01 -05:00
Cullen Watson	5d3df732e6	docs: readme	2024-03-12 20:46:25 -05:00
Kellen Mace	86f858e06d	Update scrape_jobs() parameters info in readme (#130 )	2024-03-12 20:45:13 -05:00
Cullen	1089d1f0a5	docs: readme	2024-03-11 21:30:57 -05:00
Cullen	3e93454738	fix(indeed): readd param	2024-03-11 21:23:20 -05:00
Cullen Watson	0d150d519f	docs: readme	2024-03-11 14:52:20 -05:00
Cullen Watson	cc3497f929	docs: readme	2024-03-11 14:45:17 -05:00
Cullen Watson	5986f75346	docs: readme	2024-03-11 14:41:12 -05:00
VitaminB16	4b7bdb9313	feat: Adjust log verbosity via verbose arg (#128 )	2024-03-11 14:38:44 -05:00
Cullen Watson	80213f28d2	chore: version	2024-03-11 09:43:12 -05:00
Cullen Watson	ada38532c3	fix: indeed empty location term	2024-03-11 09:42:43 -05:00
Cullen Watson	3b0017964c	fix: indeed empty search term	2024-03-11 09:21:11 -05:00
VitaminB16	94d8f555fd	format: Apply Black formatter to the codebase (#127 )	2024-03-10 23:36:27 -05:00
Cullen Watson	e8b4b376b8	docs: readme	2024-03-09 13:40:34 -06:00
Cullen Watson	54ac1bad16	docs: readme	2024-03-09 01:49:05 -06:00
Cullen Watson	0a669e9ba8	enh: indeed more fields (#126 )	2024-03-09 01:40:01 -06:00
gigaSec	a4f6851c32	Fix GlassDoor Country Vietnam(#122 )	2024-03-04 17:35:57 -06:00
troy-conte	db01bc6bbb	log search updates, fix glassdoor (#120 )	2024-03-04 16:39:38 -06:00
Cullen Watson	f8a4eccc6b	Remove pandas warning (#118 )	2024-02-29 21:30:56 -06:00
Cullen Watson	ba3a16b228	Description format (#107 )	2024-02-14 16:04:23 -06:00
Cullen Watson	aeb1a50d2c	fix job type search (#106 )	2024-02-12 11:02:48 -06:00
VitaminB16	91b137ef86	feat: Ability to query by time posted for linkedin, indeed, glassdoor, ziprecruiter (#103 )	2024-02-09 14:02:03 -06:00
Cullen Watson	2563c5ca08	enh: Indeed company url (#104 )	2024-02-09 12:05:10 -06:00
Cullen Watson	32282305c8	docs: readme	2024-02-08 18:13:19 -06:00
Cullen Watson	ccbea51f3c	docs: readme	2024-02-04 09:25:10 -06:00
Cullen Watson	6ec7c24f7f	enh(linkedin): search by company ids (#99 )	2024-02-04 09:21:45 -06:00
Cullen Watson	02caf1b38d	fix(zr): date posted (#98 )	2024-02-03 07:20:53 -06:00
Cullen Watson	8e2ab277da	fix(ziprecruiter): pagination (#97 ) * fix(ziprecruiter): pagination * chore: version	2024-02-02 20:48:28 -06:00
Cullen Watson	ce3bd84ee5	fix: indeed parse description bug (#96 ) * fix(indeed): full descr * chore: version	2024-02-02 18:21:55 -06:00
Cullen Watson	1ccf2290fe	docs: readme	2024-02-02 17:59:24 -06:00
Cullen Watson	ec2eefc58a	docs: readme	2024-02-02 17:58:15 -06:00
Cullen Watson	13c7694474	Easy apply (#95 ) * enh(glassdoor): easy apply filter * enh(ziprecruiter): easy apply * enh(indeed): use mobile headers * chore: version	2024-02-02 17:47:15 -06:00
Cullen Watson	bbe46fe3f4	enh(glassdoor): easy apply filter (#92 )	2024-02-01 19:42:24 -06:00
Cullen Watson	b97c73ffd6	fix: clean description (#88 )	2024-01-28 21:50:41 -06:00
Cullen Watson	5b3627b244	enh: full description param (#85 )	2024-01-22 20:22:32 -06:00
Cullen Watson	2ec3b04777	fix(ziprecruiter): init cookies (#82 )	2024-01-12 12:28:35 -06:00
Harish Vadaparty	89a5264391	add long scrape example (#81 )	2024-01-12 12:24:00 -06:00
Cullen Watson	a7ad616567	fix: linkedin no results (#80 )	2024-01-10 14:01:10 -06:00
cullenwatson	53bc33a43a	chore: version	2024-01-09 19:33:56 -06:00
Cullen Watson	22870438c7	linkedin fix delays (#79 )	2024-01-09 19:32:51 -06:00
Cullen Watson	aeb93b99f5	Update pyproject.toml	2024-01-03 12:04:50 -06:00
Cullen Watson	a5916edcdd	fix(glassdoor): add retry adapter (#77 )	2024-01-03 12:04:32 -06:00
Augusto Gunsch	33d442bf1e	Add czech to Indeed (#72 )	2023-12-02 02:42:54 -06:00
Zachary Hampton	6587e464fa	Update README.md	2023-11-30 11:49:31 -07:00
Vincent Yan	eed7fca300	Get full indeed description (#70 )	2023-11-27 15:00:36 -06:00
Faraz Khan	dfb8c18c51	include location with 3 parts (#69 )	2023-11-10 16:59:42 -06:00
Faraz Khan	81f70ff8a5	added salary data for linkedin (#68 )	2023-11-09 14:57:15 -06:00
Cullen Watson	cc9e7866b7	fix linkedin bug & add linkedin company url (#67 )	2023-11-08 15:51:07 -06:00
Zachary Hampton	a2c8fe046e	Update README.md	2023-11-06 22:13:19 -07:00
Cullen Watson	2b7fea40a5	[fix] glassdoor duplicates	2023-10-30 20:29:55 -05:00
Cullen Watson	d37f86e1b9	[fix] glassdoor location	2023-10-30 20:19:56 -05:00
Cullen Watson	0302ab14f5	glassdoor keywords	2023-10-30 20:07:31 -05:00
Cullen Watson	3f2b582445	add glassdoor (#66 )	2023-10-30 19:57:36 -05:00
Cullen Watson	93223b6a38	bug fix	2023-10-30 13:57:23 -05:00
Cullen Watson	e3fc222eb5	readd proxy support for zip (#64 )	2023-10-29 08:54:56 -05:00
Cullen	b303b3f841	chore: version	2023-10-28 16:58:32 -05:00
Cullen	1a0c75f323	chore: version	2023-10-28 16:54:04 -05:00
Cullen	e2f6885d61	chore: format	2023-10-28 16:52:05 -05:00
Cullen	8d65d1b652	[chore] version	2023-10-28 16:43:44 -05:00
Cullen	216d3fd39f	ziprecruiter: 5s delay	2023-10-28 16:41:32 -05:00
Cullen Watson	d3bfdc0a6e	ziprecruiter api (#63 )	2023-10-28 16:17:28 -05:00
Cullen Watson	ba5ed803ca	use ziprecuriter api (#62 )	2023-10-28 15:51:29 -05:00
Cullen Watson	ff1eb0f7b0	[docs] update readme	2023-10-18 14:32:21 -05:00
Cullen Watson	f2cc74b7f2	Fix Indeed exceptions on parsing description	2023-10-18 14:25:53 -05:00
Cullen Watson	5e71866630	[docs] link change	2023-10-18 11:18:03 -05:00
Zachary Hampton	4e67c6e5a3	Update README.md	2023-10-17 20:22:56 -07:00
Cullen Watson	caf655525a	docs: update readme	2023-10-10 11:54:14 -05:00