Issue #9: How long will humans remain part of the decision-making process?

👩‍✈️On the menu

The digitization of aviation has come a long way.

From anti-lock brakes, to fire warnings and autopilots. It is unthinkable for an airline to operate aircraft that are not stuffed with numerous computer systems to enhance the safety and efficiency of the flight.

While celebrating the benefits of digitization, it is also good to look at the risks. This issue of Airline Food for Thought will do exactly that.

• Starter: The business rules rule

• Main course: Adding complexity

• Dessert: Always keep the end goal in mind

🍜Starter

The business rules rule

On-board computer systems do two things: inform the pilot and make decisions on behalf of the pilot.

While the former has contributed to the high standard of aviation safety, we will discuss the latter. That is because aircraft manufacturers keep adding digitized systems to their aircraft with the argument to improve safety and efficiency.

When you talk about making automated decision-making, you have to talk about business rules. Whether you will look into Garmin's Autoland, Xwing's Superpilot or the Reliable Autopilot from Reliable Robotics, what you will see are a lot of business rules.

Business rules are a standardized approach to decision-making and always follow the same sequence:

• IF an event happens

• WHEN being in a certain state

• THEN the following action will be executed

One of the many business rules in the text-editor of your laptop is:

• IF you want to shut down a text document

• WHEN there are unsaved changes in the text

• THEN a pop up screen appears asking if you would like to save the document

The logic is simple and can also be applied to the autopilot in an aircraft:

• IF the aircraft is below the glide slope during an approach

• WHEN the autopilot is engaged in approach mode

• THEN the autopilot will put the aircraft back on the correct glide slope by lowering the rate of descent.

And there are many more business rules in systems varying from the auto throttle to the oven in the galley to the smoke detector in the toilet. All these business rules (as part of an automated system) have contributed to making aviation the safe way of traveling as we know it today.

🍲 Main course

Adding complexity

Business rules are not that complex when looking at it one at a time. The difficulty comes when you have to combine thousands of them as the same time.

Lucky for them, pilots in modern jet aircraft are not required to execute all the business rules. The computer systems will do that for them. What is required of pilots is to constantly monitor these systems and the decisions they make.

Because it can happen that the computers make the wrong decision.

It could be that the wrong numbers were inserted in the FMS or autopilot. Then the computer didn't make the mistake, but the pilot still has to correct the outcome. Or maybe the system has a technical problem. When that happens, it is the pilot's job to correct the decisions that were automatically made. This can be done for example by pushing the autopilot override-switch or by correcting the data in the FMS.

This synergy between men and machines has proven to be an almost flawless combination.

However, the flawlessness of this synergy is being challenged. The increased dependability on digital systems is making it more difficult for the pilots to operate the aircraft. The more systems are added to an aircraft, the more business rules are being (semi-)automatically executed. It is becoming increasingly difficult to monitor all the automated decisions and to know which decision was wrong when an unexpected situation occurs.

We have seen that happening with the two lethal accidents involving Boeing 737 MAX. Because of the increased seize of the fuselage and the engines of the new 737, the balance of the aircraft was different from the older models. Boeing created the Maneuvering Characteristics Augmentation System (MCAS) to automatically correct the attitude of the aircraft when an imbalance was about to happen (you can see the business rule being typed in the software code as you read).

Adding the MCAS adds complexity to the situational awareness of the pilots. Now more systems are involved in controlling the aircraft, making it more difficult for the pilots to judge which system is making the error. The accidents in the 737 MAX were caused by the pilots being unaware that this additional system was making the wrong decisions.

The difficulty for the pilots is that they have to operate a system that maybe is becoming too complex to be operated by people.

Fortunately, the number of incidents based on difficulties in the interface between on-board systems and their operators are rare. However, the fact that the number of computerized systems on board is constantly increasing makes it important to be aware of the risks.

The development process

The difficulty is that during a flight, two or three pilots are responsible for operating systems that took hundreds of people and many years to build. Engineers on the ground spend countless hours writing software code, creating business rules. When all goes well these business rules are a blessing. When things go wrong, they hide behind an increasingly thick disguise.

Even though thorough quality assurance processes are in place, mistakes can still be implemented. As happened with the MCAS. While the number and complexity of on-board systems is increasing, it becomes increasingly difficult for pilots to monitor and understand the decisions the aircraft makes.

More data makes it more complex

The developments towards autonomous flying will increase the communication between the aircraft and its environment. This can be vision-based as with Xwing's Superpilot or based on data links with other aircraft or ground based systems.

With the increase of real-time data that the aircraft receives based on external sources, the pilot could start to wonder if his or her situational awareness is as good as the system's. A computer can monitor other aircraft, weather and altitude all at the same time, while a person needs to check it one by one. Will the pilot understand a sudden change in attitude or direction by the autopilot? Will the pilot intervene or think "the computer probably knows best"?

It is slowly becoming the main task of human pilots to correct for the mistakes during the flight that the humans creating the system have failed to correct before the flight. This is not meant to blame the software development team when something goes wrong. However, computers only execute the business rules that people have put in place. Even tough Artificial Intelligence sounds like the computer has an independently functioning brain, this is not the case. It is an ecosystem of many people that has created that intelligence.

🍦 Dessert

Always keep the end goal in mind

To what extend can you expect a single person to have the responsibility over the actions executed by such a system? How long will humans remain part of the decision-making process? A question that is difficult to answer right know, but will be becoming relevant in the years ahead.

The difficulty is not the ability of computer systems being able to fly an aircraft. The proof is already there. The problem will be how to operate countless aircraft with hundreds of passengers at the same time. When will airlines decide to leave out the on-board pilot? Are passengers willing to fly on a plane without a pilot? Or do they rather have a pilot on board even tough the digitization of the aircraft is at a level that a he or she cannot be held accountable for its actions?

With the development of systems like the Reliable Autopilot and Autoland, the digitization of aviation creates a lot of excitement. Especially with me. Because digitization is always a means to an end and never the end itself, it is important to keep the end goal in mind. And the goal of aviation is to safely and efficiently transport people. So far, digitization has significantly contributed to that goal and probably will be doing so for a long time to come. Along the road, we as people need to keep asking ourselves to what extend new technology will help us get closer to the ultimate form of aviation. Sometimes we will have to correct the technology and sometimes the technology will correct us and our assumptions. I believe that this synergy will provide us with some stunning developments in the years to come. And as with many developments, the end phase is not the most difficult, it is the transition in between.

Thank you for getting all the way down to the end of this issue of Airline Food for Thought! If you have any questions or suggestions you can contact me via [email protected] or send me a message via LinkedIn.For more about me, visit giel.io