Skip to content

Commit History

Technical log of changes made to the project, ordered newest first.

March 2026

docs: restructure nav, add Mermaid click-to-enlarge

  • Renamed doc files to logical names (introduction, implementation, classes, database)
  • Added click-to-enlarge overlay for all Mermaid diagrams
  • Flattened nav structure, split into developer/management docs

docs: add MkDocs site, GitHub Actions deploy workflow

  • Set up MkDocs 1.6.x with Material theme
  • Added Cloudflare Pages deployment for private repo hosting
  • Created requirements-docs.txt with pinned versions

docs: add README and JobsAustria pipeline documentation

  • Added README.md with project overview
  • Created data flow sequence diagram
  • Created Stage 2 class diagram for JobsAustriaCacheSynchronizer
  • Created ERD with real schema and data lifecycle table
  • Created implementation details doc

pipeline: JobsAustria cache synchronizer and detail enrichment

  • Built JobsAustriaCacheSynchronizer — two-queue design, links scrape_cache → jobs via url_hash
  • Built JobsAustriaDetailsETL + PortalRouter — fires Apify detail actors in batches of 100, max 3 concurrent
  • Confirmed jobs_austria_cache_key_sync.py is a duplicate and safe to delete

pipeline: JobsAustria ETL cache import and payload processing

  • Built JobsAustriaETLCache — async producer/consumer, full refresh scraper using asyncio.gather
  • Built JobsAustriaCacheProcess — seeds jobs table from scrape_cache payload JSON