Airline Crew Scheduling Optimization

Executive Summary

Problem

Airlines have the enormous challenge of managing delayed flights, sick crew members, and other external factors that constantly disrupt their crew rosters. The client, a large European Airline, has a 24/7 control centre where a crewing team continuously repairs the roster to keep every flight legally crewed.

Human operators are very good at their job, but during busy periods (school holidays, and especially the summer months) the volume of disruption exceeds what the team can manually keep up with. When that happens, flights end up violating the legal rules around crewing. The consequences are further delays, cancellations, and potential punitive measures from the aviation regulator and trade unions — up to and including grounding the fleet.

After a summer with an unacceptable rise in such violations, the client committed to a set of mitigation measures with the regulator. They wanted a solution that can provide timely, data-driven recommendations to the crewing team, and it had to be live before the next summer season.

Solution

We built a Crew Violation Recommender: a custom optimization engine that resolves multiple roster violations at the same time, rather than triaging each one in isolation.

The guiding principles were:

1. Keep the human in the loop. The tool recommends; the crewing officers decide.
2. Make as few changes as possible to the existing roster.
3. Take the tedious, time-consuming violations off the operators’ plate so they can focus on the hard problems.

The solution runs as an end-to-end Databricks pipeline that ingests the latest roster data and produces a fresh set of recommendations every 30 minutes, served to the crewing team through a purpose-built UI with a feedback mechanism.

The full engagement — from understanding the problem to production and handover — was delivered in 5 months, in time for the summer season.

Outcome

1. During user acceptance testing on a heavily disrupted day, the tool resolved 5 uncrewed flights in the time it took a crewing officer to manually fix one — a 5x improvement in that instance.
2. Some of its recommendations were better than the ones the experts had actually used, for example by avoiding Short Notice Changes, which carry a direct financial cost.
3. Recommendations are explainable: when one cannot be implemented, the tool can tell you why, which proved extremely useful for understanding the MVP’s limitations.
4. The crewing team got a self-serve tool: a Databricks pipeline refreshing every 30 minutes, a UI for reviewing and applying recommendations, and logging of which recommendations were accepted or rejected.

---

The challenge

1. An enormous solution space

The scale of the problem is what makes it interesting: over 1,700 flights and 7,000 active crew members per day, across a network of 30 bases, governed by 150+ legal rules. Every crew member has a roster of flights, ground duties, training, and days off, and moving a single duty can cascade into new violations elsewhere. Several stakeholders believed the approach was technically infeasible.

The honest first step was aggressive scoping. The MVP focused on cabin crew at a single major base, a prioritized subset of the rules, and recommendations for today, tomorrow, and the day after. Separating the MVP from the nice-to-haves was genuinely hard, because so many of the rules and features had interlinked dependencies. Rules we could not fit into the optimizer in time were not ignored — they were handled by a separate component (more on that below).

2. Choosing the right algorithm

With this many rules and this much scale, exact optimization methods were never going to return an answer within an operationally useful time window. We implemented an altered version of the ALNS (Adaptive Large Neighborhood Search) metaheuristic.

The core idea:

• The roster data is ingested into an interlinked structure of Python data objects: CrewRoster (one crew member’s duties), Sector (a single flight), and State (a full feasible assignment of crew to flights).
• Individual removal and insertion heuristics are paired up into actions, and the probability of each action being chosen is updated over the course of the optimization.
• We deliberately designed the heuristics to reflect the actions crewing officers take themselves — swapping routes, pulling crew from standby — which made the output feel familiar rather than alien.
• The cost function is a weighted combination of satisfying key constraints, minimizing uncrewed flights, minimizing violations, and minimizing changes to the roster.

Constraints are enforced both at ingestion and during optimization: is the crew member qualified for this flight? Is this duty in scope to be moved? Will the new duty overlap with an existing one, or create a new rest issue?

3. When the optimizer can’t know all the rules

The optimizer only encodes a prioritized subset of the 150+ rules. Shipping a recommendation that fixes one violation while quietly creating another would destroy the operators’ trust in the tool very quickly.

The solution was a separate violation checker component. It evaluates the optimized rosters against a wider rule set than the optimizer itself handles and filters the outputs down to only those that hold up. It also powers the explainability: when a recommendation cannot be implemented, the checker can point at the rule that kills it — effectively a root-cause analysis for free.

4. Running fast enough on Databricks

The solution had to run in Databricks, run fast, and run reliably. An extra constraint from the legacy world: the crew management system has limited programmatic access, and the available data runs 30–45 minutes behind live. There is no point spending an hour optimizing a roster that no longer exists, so the whole pipeline is built around a 30-minute cadence.

Metaheuristics are stochastic, so a single run can get unlucky. We used Ray to run 16 optimizers in parallel, which greatly increased the likelihood of finding a good solution within the time budget. Running Ray inside Databricks was new territory in the client’s environment, and getting it stable took some pioneering.

CI/CD and MLOps practices did a lot of quiet heavy lifting here: automated testing and deployment meant a bugfix driven by user feedback could be in production in about an hour. In a solution space that changes this quickly, that turnaround was a requirement, not a luxury.

5. Keeping the humans on side

The tool is only useful if the crewing team actually uses it. We worked directly with the crewing officers throughout: UI mock-ups early on helped them visualize how the product would fit their existing process, and producing real outputs for review as early as possible kept them invested.

The UI (built with Plotly) presents each recommendation as a series of simple swaps between crew members, alongside the business context of the violation being solved. Operators can accept or reject each part of the output, and that feedback is logged — which let us rapidly prioritize the developments that mattered. The product went from producing limited valid recommendations in mid-June to out-pacing the crewing team on a heavily disrupted day two weeks later.

The solution

The final architecture is a deliberately simple sequential Databricks workflow:

1. Ingestion: PySpark pipelines pre-process the crew management system data into the Python data object structure.
2. Optimization: 16 parallel ALNS optimizers (vanilla Python, parallelized with Ray) search for rosters that maximize crewed flights while minimizing violations and changes.
3. Violation checking: candidate rosters are filtered against the wider rule set to maximize legality and usefulness.
4. Serving: post-processed recommendations, broken into contextualized sub-solutions, are served to the crewing team through the UI every 30 minutes, with accept/reject feedback logged for evaluation against the agreed key metrics.

Data latency remains the honest limitation: recommendations are only as fresh as the 30–45 minute old data they are built on, and in tactical optimization that is a serious bottleneck. Solving it sits upstream of this project, so we deferred that problem to the future.