How to Monitor Website Content Changes in Python
This script fetches a webpage's content, computes its SHA-256 hash, and compares it with the last stored hash to detect and alert on changes.
pip install requests
Python code
27 linesimport time
import hashlib
import requests
from pathlib import Path
def fetch_content_hash(url: str) -> str:
response = requests.get(url, timeout=10)
response.raise_for_status()
return hashlib.sha256(response.text.encode()).hexdigest()
def monitor_website(url: str, check_interval: int = 60):
hash_file = Path("last_hash.txt")
if hash_file.exists():
last_hash = hash_file.read_text().strip()
else:
last_hash = ""
current_hash = fetch_content_hash(url)
if current_hash != last_hash:
print(f"ALERT: Content changed on {url}")
hash_file.write_text(current_hash)
else:
print(f"No changes detected on {url}")
if __name__ == "__main__":
target_url = "https://example.com"
print("Starting website monitor (one check)...")
monitor_website(target_url, check_interval=60)
Output
Starting website monitor (one check)...
No changes detected on https://example.com
ALERT: Content changed on https://example.com
How it works
The script uses requests.get() to download the page content and hashlib.sha256 to compute a content hash. It persists the last hash in a local file last_hash.txt, allowing detection of changes across runs. On each check, it compares the current hash to the stored one and prints an alert if they differ. This approach is efficient because hashing the full page text reduces storage and comparison overhead.
Common mistakes
- Not including `raise_for_status()` which can silently miss HTTP errors and return a wrong hash.
- Forgetting to handle the case where the hash file doesn't exist on the first run.
- Using a weak hash like MD5 which could produce collisions in adversarial scenarios.
- Setting a very short check interval that may overwhelm the server or get your IP blocked.
Variations
- Use `time.sleep(check_interval)` in a `while True` loop to run checks continuously.
- Instead of hashing the whole page, hash specific elements (e.g., a known dynamic section) for more targeted monitoring.
Real-world use cases
- Monitor a competitor's pricing page and trigger an internal alert when prices are updated.
- Watch a job board or government website for new postings and notify a Slack channel.
- Track documentation or policy pages for changes that require a compliance review.
Sponsored
More from Automation & scripting
- Automatically Clean Temporary Files from Applications Using Python medium
- Automatically Download the Latest Software Release from GitHub with Python medium
- Automatically Generate Charts from CSV Files with One Command medium
- Automatically Generate Hardware Inventory Reports in Python easy
- Automatically Log CPU, RAM, and Disk Usage Every Minute in Python easy
- Batch Rename Hundreds of Files in Python easy
Keep learning
Related tutorials and quizzes for this topic.