maurosoria/dirsearch ⭐ 14,006

# Coding Agent Guide (dirsearch)

This repository contains **dirsearch**, a web path discovery tool. Use this guide to keep changes aligned with project expectations and release workflows.

## Scope
- These instructions apply to the entire repository.
- If a subdirectory contains its own `AGENTS.md`, that file takes precedence for that subtree.

## Project overview (high-level map)
- **Entrypoint**: `dirsearch.py` is the CLI entry for running scans.
- **Core flow**: `lib/controller/controller.py` orchestrates scans, sets up reports, handles sessions, and manages runtime flow.
- **Networking**: `lib/connection/requester.py` (sync) and `lib/connection` modules manage HTTP requests, proxies, auth, rate limiting, and DNS caching.
- **Reporting**: `lib/report/` and `lib/report/manager.py` manage output formats (plain, JSON, XML, CSV, HTML, SQLite, MySQL, PostgreSQL, etc.).
- **Sessions**: session persistence is handled by `lib/controller/session.py`, with default session paths configured in `lib/core/settings.py`.
- **Builds**: PyInstaller build files live under `pyinstaller/`, with CI workflows under `.github/workflows/`.
- **Docker**: `Dockerfile` provides a minimal containerized entrypoint.

## Directory structure (quick map)
- `lib/`: Core application code (modules grouped by responsibility).
  - `lib/core/`: settings, options, data, and shared helpers.
  - `lib/controller/`: orchestration and session handling.
  - `lib/connection/`: HTTP/DNS/networking stack.
  - `lib/parse/`: parsing helpers (URLs, raw requests).
  - `lib/report/`: reporting formats and output handlers.
  - `lib/utils/`: utility modules shared across the codebase.
  - `lib/view/`: terminal output and UI helpers.
- `db/`: bundled wordlists and categories.
- `tests/`: test data and unit tests.
- `sessions/`: default session output location for source runs.
- `static/`: static assets (logos).
- `.github/workflows/`: CI/CD, security scanning, and packaging workflows.

## Code style and architecture
- Prefer **pythonic** code: clear naming, readable structure, and small, testable functions.
- Use **polymorphism and classes** where it improves readability or flexibility (especially within `lib/`), avoiding unnecessary complexity.
- Treat `lib/` as a modular framework: keep boundaries clean, use explicit interfaces, and avoid cross-layer leakage.
- Add **comments for edge cases** so behavior is clear to future maintainers.

## When you change X, check Y (dependency map)
Use this section to keep side effects aligned and to avoid missing required updates.

### CLI / options / config
- If you add or change CLI options, update:
  - `README.md` options/usage docs.
  - `config.ini` defaults (when relevant).
  - Any tests or examples referencing the old flags.

### Output formats / reports
- If you add or change report formats:
  - Update `lib/report/manager.py` to register the handler.
  - Confirm the format appears in README output examples if documented.
  - Ensure any CI tests that generate reports still pass (see `.github/workflows/ci.yml`).

### Sessions
- If you change session content or schema:
  - Update `lib/controller/session.py` serialization/deserialization logic.
  - Update any references to default session locations in docs (see `README.md`).
  - Consider backward compatibility for older session formats.

### Networking / request pipeline
- If you modify request logic, proxies, auth, or rate limiting:
  - Review `lib/connection/requester.py` and `lib/connection/dns.py`.
  - Validate behavior with relevant CLI options (proxy, auth, random agents, max rate).
  - Consider updates to tests or example commands in CI.

### Controller / workflow behavior
- If you change scan orchestration or run flow:
  - Review `lib/controller/controller.py` for session handling, callbacks, and report preparation.
  - Ensure that report saving and session export still operate as expected.

### Build & release artifacts (PyInstaller)
- If you add modules or dependencies that must be bundled:
  - Update PyInstaller hidden imports or data in:
    - `pyinstaller/dirsearch.spec` (preferred), and
    - GitHub Actions PyInstaller workflows under `.github/workflows/` (Linux/macOS/Windows).
  - Verify `pyinstaller/build.sh` still produces a working binary.
  - If you change outputs or binary names, update the release workflow (`pyinstaller-release-draft.yml`).

### Docker
- If you add OS-level dependencies or change runtime behavior:
  - Update `Dockerfile` accordingly.
  - Ensure new files are included in the container build context.

### CI / GitHub Actions
- If you change dependencies, CLI usage, or tests:
  - Update `.github/workflows/ci.yml` to keep inspection steps and command flags in sync.
  - Consider whether CodeQL, Semgrep, Docker build, or PyInstaller workflows need updates.

### Adding a new feature or major behavior
- Update docs (`README.md`, CLI examples, and any new flags).
- Ensure reports/session outputs include the new data if applicable.
- Verify CI commands still cover the new feature path.
- Consider whether Docker, PyInstaller, or release workflows need updates for new dependencies or files.

## Current automation (quick reference)
- **CI / Inspection**: `.github/workflows/ci.yml` runs CLI scans, `testing.py`, lint (flake8), and codespell.
- **Security**: CodeQL (`codeql-analysis.yml`) and Semgrep (`semgrep-analysis.yml`) run on PRs/pushes.
- **Docker**: `docker-image.yml` builds the Docker image on pushes and PRs to `master`.
- **PyInstaller**: platform builds and draft release workflows live under `.github/workflows/pyinstaller-*.yml`.

## Testing guidance
- For logic changes, try:
  - `python3 testing.py`
  - `python3 -m pytest` (if you touch tests or add new ones)
- For CLI changes, run a short scan against a sample target:
  - `python3 dirsearch.py -w ./tests/static/wordlist.txt -u https://example.com -q`

## Communication checklist (for summaries / PRs)
- What changed and why.
- Any updates to docs, CLI flags, or output formats.
- Whether sessions or reports were affected (and if a migration is required).
- Tests executed and their results.