JobsAustriaDetailsETL

File: jobs_austria_details_scraping.py
Inherits: —

Fetches job URLs from the DB that still need detail extraction, fires them at the ams_details Apify actor in batches of URL_BATCH_SIZE, then streams results back and persists them.

Class Diagram

classDiagram
    class JobsAustriaDetailsETL {
        +__init__()
        -_create_engine()
        +async connect()
        +async disconnect()
        -async _fetch_pending_urls() list
        -_build_configs() list
        -_start_actors() list
        -async _drain_generator()
        +async produce_apify_results()
        +async consume_apify_results()
        +async produce_and_consume()
        +async run()
    }

Hold "Ctrl" to enable pan & zoom

---

Methods

Method	Parameters	Returns
__init__()	—	—
_create_engine()	—	—
connect()	—	—
disconnect()	—	—
_fetch_pending_urls()	—	list
_build_configs()	urls	list
_start_actors()	configs	list
_drain_generator()	generator: ApifyGenerator	—
produce_apify_results()	—	—
consume_apify_results()	—	—
produce_and_consume()	—	—
run()	—	—

Attributes

No class-level attributes.