Improve documentation - structure & content revision #1752

Summary

The SDK and API Client documentation is currently undergoing a revision and structural improvements led by the docs & DX team. We should apply the same approach to Crawlee for Python documentation.

The existing content is generally good, but the documentation would benefit from:

1. Unified and improved structure — reorganize sections for better discoverability and navigation
2. Content audit — identify weak spots, gaps, and areas that need more detail or clarity
3. Undocumented features — find features that are missing from the docs entirely
4. Consistency — align style, terminology, and patterns across all doc pages

Proposed approach

• Review the current documentation structure and propose a better, more organized layout
• Use AI agents to systematically scan the codebase and identify:
  • Features / APIs that are not documented
  • Inconsistencies between code and docs
  • Areas where examples are missing or outdated
• Coordinate with the docs & DX team to align on structure and style conventions used in the SDK / API Client docs revision
• Incrementally improve content based on the audit findings

References

• Crawlee for Python docs: https://crawlee.dev/python/docs/quick-start
• SDK docs revision (ongoing by docs & DX team) — follow the same patterns & conventions