RawRateModel — Developer Reference

Developer reference for RawRateModel used for the Opera chain (daily rates data).

File: etl_lib/models/raw/RawRateModel.py

Constructor Parameters

Parameter	Type	Description
`job_context`	`JobContext`	Job context with Spark, config and catalog
`database`	`GlueDatabases`	Default: `GlueDatabases.RAW`
`table`	`RawTables`	Default: `RawTables.DAILY_RATES`
`df`	`DataFrame`	Optional DataFrame to use instead of reading from the catalog
`overwrite_partitions`	`bool`	Whether to overwrite partitions when writing

Purpose & Usage

RawRateModel is a simple wrapper over the Model class that points to the daily_rates ingest table. The OperaIngesterTask writes daily rates to /ingest under the daily_rates logical table which OperaCrawlerTask will read.

Use RawRateModel when joining rates to reservations during the cleaning step (e.g., CleanOperaTask).

Example


from etl_lib.models.raw.RawRateModel import RawRateModel
 
model = RawRateModel(job_context=job_context)
df = model.get()

Back to models: [/processes/models](/processes/models)