🔎 Introduction
Open-Source Intelligence (OSINT) is a powerful method for gathering information from publicly available sources. Whether you’re an ethical hacker, cybersecurity professional, journalist, or investigator, Python can significantly enhance your OSINT capabilities. With its extensive libraries and automation potential, Python allows you to efficiently scrape, analyze, and visualize data.
📘 Get started with OSINT using our eBook: Python for OSINT
🚀 Why Use Python for OSINT?
Python is an ideal choice for OSINT due to:
- Automation – Saves time by automating data collection.
- Web Scraping – Extracts data from websites easily.
- API Integration – Interacts with social media, search engines, and databases.
- Data Analysis – Cleans, processes, and visualizes gathered intelligence.
- Anonymity & Privacy – Works with proxy tools and anonymization services.
🔥 Essential Python Libraries for OSINT
- Requests – Fetch web pages and interact with APIs.
import requests
response = requests.get("https://example.com")
print(response.text)
- BeautifulSoup – Scrape and parse HTML data.
from bs4 import BeautifulSoup
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.text)
- Selenium – Automate web browsing and bypass restrictions.
from selenium import webdriver
driver = webdriver.Chrome()
driver.get("https://example.com")
- Shodan – Query IoT and network intelligence.
import shodan
api = shodan.Shodan("YOUR_API_KEY")
results = api.search("apache")
print(results["matches"])
- Whois – Retrieve domain information.
import whois
domain_info = whois.whois("example.com")
print(domain_info)
- theHarvester – Gather emails, subdomains, and more.
theHarvester -d example.com -b google
🌍 OSINT Techniques Using Python
1️⃣ Gathering Social Media Data
Use APIs like Twitter, Facebook, or Instagram to extract user data.
import tweepy
auth = tweepy.OAuthHandler("API_KEY", "API_SECRET")
api = tweepy.API(auth)
tweets = api.user_timeline(screen_name="codelivly", count=5)
for tweet in tweets:
print(tweet.text)
2️⃣ Tracking IP Addresses
import requests
ip = "8.8.8.8"
response = requests.get(f"https://ipinfo.io/{ip}/json")
print(response.json())
3️⃣ Email Enumeration
import requests
email = "test@example.com"
response = requests.get(f"https://api.hunter.io/v2/email-verifier?email={email}&api_key=YOUR_API_KEY")
print(response.json())
4️⃣ Extracting Hidden Data from Websites
Search for exposed admin panels, directories, and sensitive files.
urls = ["robots.txt", "sitemap.xml", ".git/config"]
for url in urls:
response = requests.get("https://example.com/" + url)
if response.status_code == 200:
print(f"Found: {url}")
🛠️ Automating OSINT with Python
Automate repetitive OSINT tasks by creating scripts that:
- Monitor websites for changes
- Track social media activities
- Scrape and store data from multiple sources
- Generate intelligence reports
Example: Automating Social Media Monitoring
import tweepy
from datetime import datetime
def monitor_twitter(username):
tweets = api.user_timeline(screen_name=username, count=10)
for tweet in tweets:
print(f"[{datetime.now()}] {username}: {tweet.text}")
monitor_twitter("codelivly")
🔒 Staying Anonymous While Conducting OSINT
To protect your identity:
- Use Tor (
stem
library for Python) - Employ VPNs & Proxies (
requests[socks]
) - Rotate User-Agents & IPs
Example: Using Tor with Python
import requests
proxies = {"http": "socks5h://127.0.0.1:9050", "https": "socks5h://127.0.0.1:9050"}
response = requests.get("http://check.torproject.org", proxies=proxies)
print(response.text)
📌 Conclusion
Python makes OSINT investigations faster, smarter, and more efficient. With the right libraries and techniques, you can extract valuable intelligence from the web while staying anonymous.
📘 Learn more with our detailed eBook: Python for OSINT
🚀 Happy Hunting! 🕵️♂️