Executive Summary
Problem
The client, a large UK private hire (taxi) operator, faces the same problem every marketplace does: supply and demand are never in balance. On a rainy Friday night there are more passengers than driver-partners, bookings go unfulfilled and customers churn to competitors. On a quiet Tuesday afternoon drivers sit idle and earn nothing.
Surcharges (surge pricing) are the classic lever to correct this imbalance, but the client was setting them manually: static schedules, configured per city, based on the operators’ gut feel. There was no data-driven way to know what the right price was for a given place, time and market condition, and no way to react when conditions changed.
Solution
We built an end-to-end dynamic pricing solution:
- A cloud data platform that replicates the dispatch system, call centre and mobile app data in near real time.
- Supply and demand forecasting models that predict the market conditions for every city zone, hourly, 7 days ahead.
- A reinforcement learning model that learns the optimal surcharge for each market condition and recommends prices daily.
- A human-in-the-loop review workflow, so business users can inspect, adjust or veto every price before it goes live.
- Automated deployment of the approved surcharges to the dispatch system, plus explainability reports that show why each price was recommended.
Outcome
During a 6-week evaluation period, the client measured a 7-10% revenue uplift (~£250k) in the regions where the model was live. The solution was rolled into a managed service which we have operated for over 3 years, with quarterly model health checks.
The challenge
1. Defining supply and demand you cannot observe
The dispatch system records completed bookings. That is satisfied demand — the easy part. The demand we actually cared about was the demand we lost: the customers who wanted a ride but didn’t get one. Same for supply: the drivers who were logged in to the app but chose not to take jobs (latent supply) are exactly the ones a surcharge is supposed to activate.
Neither of these is directly observable, so we had to derive them:
- Unsatisfied demand: unique passengers (identified by phone number) with cancelled, no-show or uncompleted app bookings, per zone per hour.
- Latent supply: drivers who performed at least one action in the app that hour but completed no bookings.
These definitions rest on simplifying assumptions that we made explicit and agreed with the client: 1 phone number = 1 user, and for the call centre data — where only call metadata was available — any call over 30 seconds counts as a booking attempt. Are these assumptions bulletproof? No. Were they good enough to give the models a consistent signal? Yes, and we documented the limitations rather than hiding them.
2. Getting the data out of an on-prem datacenter
All the source data lived in MySQL databases in the client’s on-prem datacenter, behind their VPN. We set up change data capture with Datastream, replicating 3 production databases into BigQuery in near real time (data lands within ~15-20 minutes), with the connection tunnelled through a reverse proxy VM into the cloud VPC.
The replication is faithful — sometimes too faithful. Upstream deletions are replicated as-is, so there is no recovering raw history the source system decides to drop. And the latency varies with the client’s network and server load, which put a hard floor on how “real-time” the pricing could ever be. We designed for a daily recommendation cadence instead of chasing minute-level reactivity, which the data could not support anyway.
On top of the raw data we built the warehouse models with dbt on BigQuery — supply and demand metrics, driver activity, waiting times — orchestrated through a Terraform-managed CI/CD pipeline, with data volume monitoring and automated alerting on every service in the chain.
3. Forecasting a spiky market
The reinforcement learner needs to know tomorrow’s market conditions before it can price them. So we built forecasting models that predict six supply and demand metrics per city zone, hourly, over a 168-hour horizon.
The models are autoregressive (GARCH family), one per zone and metric, retrained every day on a 2-year rolling window. They are univariate — history only, no weather feeds, no bank holiday calendars. That is a deliberate MVP trade-off we flagged to the client, not an oversight: taxi demand is dominated by strong daily and weekly seasonality, and the autoregressive models capture that well.
Forecast accuracy is monitored with volume-weighted error reports, and the managed service includes quarterly forecast reviews. An honest admission: there is no automated data drift monitoring — the quarterly human health check is the mitigation, and we documented it as such.
4. Making reinforcement learning safe enough for production
Deep RL makes great conference talks, but this system changes real prices for real passengers every day. Our priority was a learner we could trust, explain and debug.
The first design decision was tabular Q-learning over function approximation. RL theory warns about the “deadly triad” — function approximation, bootstrapping and off-policy learning together cause instability (see the bottom of the page for more details). We needed bootstrapping and off-policy learning (to pre-train from historic data), so the approximation had to go.
Keeping a Q-table honest means keeping the state space small. A naive state encoding — region, sub-region, day of week, hour, weather, holidays — blows up to 443,520 states before you even add price; nowhere near enough data to visit them all. Instead we compressed the state to what actually drives pricing: total supply and total demand levels per zone, log-transformed and binned into 10 levels each. The log-binning is admittedly a hack, but a principled one: it puts bookings, drivers and wait times on comparable scales and caps the damage from unseen extreme values.
The rest of the design followed the same philosophy:
- Actions are a small set of pre-agreed surcharge increments per region (e.g. £0.00 / £0.50 / £1.00), not a continuous price the model could run away with.
- Reward combines satisfied demand, satisfied supply and average customer wait time (in 5-minute bins), bounded between -7 and 18. Revenue is deliberately not the only signal — the client cares about fulfilling bookings, not just fulfilling expensive ones.
- Pre-training replays historic state-action-reward transitions from the warehouse, so the model launched with a sensible policy instead of learning at the customers’ expense.
- Exploration is capped: 90% of the time the model recommends its best-known price, 10% of the time it tries an alternative — and every explored price still passes through human review.
5. The last mile: humans, spreadsheets and a dispatch API
The client was not going to hand pricing over to an algorithm on day one, and rightly so. Every day the recommender writes the next day’s surcharges to a Google Sheet per region. Business users review and edit the prices, the approved ones are pushed to the dispatch system’s API, and they go live.
Yes, the front end of a machine learning system is a spreadsheet. We evaluated fancier options, but the users already lived in spreadsheets, access control was solved by the client’s existing Google workspace, and the budget was better spent on the models. We are optimizing for adoption, not for architecture diagrams. The trade-off is limited input validation, which we mitigated with locked ranges, drop-down value lists and a user guide covering the failure modes.
The dispatch system’s surge pricing API was the final hurdle: sparsely documented, with edge cases we could only discover by testing against it. To build trust in the loop, every recommendation ships with explainability graphs: the supply and demand forecast that informed the price, and a heatmap of the learned policy showing which market conditions trigger which surcharge. When a business user asks “why is Saturday night £1.00?”, the answer is a picture, not a shrug.
Extra material for the curious
The deadly triad, from Sutton and Barto’s Reinforcement Learning: An Introduction, is the observation that an RL system combining all three of the following is prone to divergence — the value estimates oscillate or blow up and no meaningful learning happens:
- Function approximation — using a neural network (or any parametric model) instead of a table to store values, necessary for very large state spaces.
- Bootstrapping — updating value estimates from other value estimates rather than waiting for complete outcomes, necessary for learning at a reasonable speed.
- Off-policy training — learning about one policy while following another, necessary for pre-training from historic data that was not generated by the model.
Remove any one of the three and stability is recoverable. We removed function approximation: by aggressively compressing the state space (log-binned supply and demand levels per zone), a plain Q-table was sufficient, and we kept the bootstrapping and off-policy pre-training that the project actually needed.