Competitive Pricing Intelligence Pipeline

The Problem

Nigerian retail businesses lack systematic visibility into competitor pricing. Price decisions are based on manual observation and gut feel, leading to suboptimal pricing strategies and missed revenue opportunities.

My Approach

I built a two-layer pipeline: Python ingests Nigerian retail competitor pricing data from HuggingFace Parquet datasets and loads it into Snowflake. dbt transforms the raw data into a dimensional star schema with SCD Type 2 snapshots for historical tracking. Power BI dashboards surface win rates, undercut analysis, and a composite threat index.

What I Built

• Python extraction module reading Parquet via hf:// protocol directly from HuggingFace
• Python transformation with price recalculation, null handling, and time feature engineering
• Snowflake ingestion via write_pandas with auto_create_table
• dbt staging view with explicit type casting and percent normalization
• 3 dimension tables (product, competitor, date) with deterministic surrogate keys
• Fact table with denormalized fields for BI convenience
• SCD Type 2 snapshot capturing every price change over time
• Custom data quality tests: uniqueness, not-null, accepted values, FK relationships
• Power BI dashboard with Win Rate, Undercut Rate, Avg Price Gap, and Threat Index
• dbt docs with interactive lineage DAG

Impact & Results