Why Leadership Development Programs Fail — and How To Do It the Right Way

Image: Pinterest

Leadership must be important. More than 20,000 books and thousands of articles have been written about the critical elements of leadership and the impact it makes on people, organizations and countries, if not the world.

Yet, today, despite the collective wisdom of centuries on this topic, confidence in our leaders is low and continues to decline. Seventy-seven percent of those polled nationwide in the U.S. say that the country now has a crisis in leadership and confidence levels have fallen to the lowest levels recorded in recent times.

According to various surveys carried out by 14CP, the Institute for Corporate Productivity, approximately 75% of organizations find their leadership development programs ineffective. And according to a Korn Ferry Real World Leaders report, only 17% of executives believe their organization has the leadership capabilities they need to achieve their goals.

For years, organizations have lavished time and money on improving the capabilities of managers and on nurturing new leaders. US companies alone spend almost $14 billion annually on leadership development. Colleges and universities offer hundreds of degree courses on leadership, and the cost of customized leadership-development offerings from a top business school can reach $150,000 a person.

Michael Beer, Magnus Finnström, and Derek Schrader in their article “Why Leadership Training Fails — and What to Do about It.” Harvard Business Review, argue “U.S. corporations spend enormous amounts of money — some $456 billion globally in 2017 alone — on employee training and education, but they aren’t getting a good return on their investment. People soon revert to old ways of doing things, and company performance doesn’t improve.”

Leadership Development Done the Right Way — ViRTUS

ViRTUS, a Vancouver-based B-Corp leadership development and coaching company, serving a wide variety of clients in North America, is an example of a consultancy that has the right formula for leadership development. ViRTUS says it helps “leaders develop the emotional intelligence and leadership capabilities they need to grow and thrive. We help senior teams create strategies that are inspiring, executable, and that drive real outcomes.” ViRTUS also leads by example, by developing their own corporate culture that develops and supports their employees and contractors in their learning, development and well-being. ViRTUS has received exceptional positive feedback from virtually all their clients.

Why Does Leadership Development Fail?

In the past two decades, 30% of Fortune 500 chief executives have lasted less than three years. Top executive failure rates as high as 75% and rarely less than 30%. Chief executives now are lasting 7.6 years on a global average down from 9.5 years in 1995. According to the Center for Creative Leadership, 38% new chief executives fail in their first 18 months on the job.

It appears the major reasons for failure has nothing to do with competence, or knowledge, or experience. Sydney Finkelstein, author of Why Smart Executives Fail, and David Dotlich and Peter C. Cairo, in their book, Why CEOs Fail: The 11 Behaviors That Can Derail Your Climb To The Top And How To Manage Them, present cogent reasons why chief executives fail, most of which have to do with hubris, ego, a lack of emotional intelligence, narcissism and psychopathy.

In a provocative article in the Harvard Business Review Blogs, Tomas Chamorro-Premuzic, author of book is Confidence: Overcoming Low Self-Esteem, Insecurity, and Self-Doubt. and his follow-up recent book, Why Do So Many Incompetent Men Become Leaders? (and how to fix it),argues more males fail as leaders than women: “The main reason for the uneven management sex ratio is our inability to discern between confidence and competence. That is, because we (people in general) commonly misinterpret displays of confidence as a sign of competence, we are fooled into believing that men are better leaders than women. In other words, when it comes to leadership, the only advantage that men have over women (e.g., from Argentina to Norway and the U.S. to Japan) is the fact that manifestations of hubris — often masked as charisma or charm — are commonly mistaken for leadership potential , and that these occur much more frequently in men than in women.

Chamorrow-Premuzic goes on to contend the paradoxical implication is that the same psychological characteristics that enable male managers to rise to the top of the corporate or political ladder are actually responsible for their downfall. In other words, what it takes to get the job is not just different from, but also the reverse of, what it takes to do the job well. As a result, too many incompetent people are promoted to management jobs, and promoted over more competent people.

Virtually anywhere in the world men tend to think that they that are much smarter than women, Chamorrow-Premuzic, argues, yet “arrogance and overconfidence are inversely related to leadership talent — the ability to build and maintain high-performing teams , and to inspire followers to set aside their selfish agendas in order to work for the common interest of the group. Indeed, whether in sports , politics or business, the best leaders are usually humble — and whether through nature or nurture, humility is a much more common feature in women than men. A study of all sectors and 40 countries, shows that men are consistently more arrogant, manipulative and risk-prone than women.” Unsurprisingly, the mythical image of a “leader” embodies many of the characteristics commonly found in personality disorders , such as narcissism and psychopathy as well as histrionic or Machiavellian personalities.

In a published media interview, Brigid Carroll, an associate professor at the University of Auckland School of Business rightly deplored what Harvard Business Review called “the great training robbery,” with billions squandered by business worldwide on ineffective leadership development programs.

Carroll, author of the book, Responsible Leadership: Realism and Romanticism, co-edited with British academic Steve Kempster, disparages what she deems the inadequacy of the traditional understanding of leadership, which she believes focuses on individual charismatic leadership. She prefers instead to see leadership as asking the big questions, like “sustainability, social responsibility, innovation, marketplace shifts, and inequality and seeking solutions through groups connecting and holding responsibilities.”

Part of the reason why leadership development programs fails is inextricably tied to our notions of what makes a good leader. Tasha Eurich, author of Bankable Leadership: Happy People, Bottom Line Results and the Power to Deliver Both argues “Though scientists spent most of the 19th century convinced that good leadership was inborn and fixed, the research of the early 20th century told a different story — that leadership is largely made. A recent study by Richard Arvey at Singapore’s NUS Business School revealed that up to 70 percent of leadership is learned. But business leaders are divided. The Center for Creative Leadership reports that 20 percent of C-level executives believe that leadership is born, and more than 28 percent believe it’s equally born and made. But, the evidence shows otherwise.”

Cynthia Kivland and Natalie King of the University of Illinois, argue leadership development programs fail because:

1. No Long-term measures prove the permanency of training. Companies require measures for their own performance, but don’t require measures on the permanency of their leadership training.

2. Lack of management support. The extent to which new skills or behavior are reinforced by management is directly proportional to the adoption of these behaviors into the culture.

3. Leaders don’t walk the talk. Organizations often believe that the degree of employee support and commitment to training is directly related to the amount of training dollars or how loud the leadership team champions a specific training program.

4. The absence of pre-screening to measure the compatibility of the trainee to the training or the training to the culture. Studies of the influence of trainees’ characteristics on training effectiveness have focused on the level of ability (Able) necessary to learn program content. Just as important are the motivational (Willingness) and environmental influences (Readiness) of the trainee and culture.

According to an MIT study of dozens of companies over two decades identifies three “pathologies” that may account for leadership failures. The first is “the ownership is power mind-set.” Older ways of managing, predominantly a command and control style persist, and these ways collide with the new realities of what makes organizations and their employees behave, which requires a system of shared accountability and responsibility. The MIT study argues that ownership and control is the wrong issue and illustrates old-world thinking. Leaving responsibility solely to the CEO or management team or HR for developing leaders is not realistic and is not successful.

The second pathology identified in the MIT study is the “productization of leadership development,” or in other words, leadership development is linked to the products of the organization, rather than the overarching problems that need to be solved. Which often leads to quick fixes. This is frequently seen as a leadership program based on a best-selling book or off-the-shelf leadership program purchased from a consultant. The result frequently ends up in the flavor of the month approach to training programs, which are often frequently forgotten in a short time.

During tough economic times, top executives decide to curtail investments in leadership development, ushering in the return of a more Darwinian model of leadership — “the cream will rise to the top.” Employees then become cynical about the company’s dedication to leadership development. High-potentials hesitate before investing their energy in developmental initiatives; some of the best walk away from the organization, and others do not reach their potential for lack of strong developmental experiences. In this scenario, there are no winners.

The third pathology that the MIT study identifies is “make believe metrics.” Most organizations require accountability for their expenditures, which is often driven by metrics. Today there are scorecards for virtually everything, including leadership development. However, the MIT study concludes, the use of metrics for the effectiveness of leadership development is leading them astray. Most metrics don’t measure the soft skills, or strategic thinking or collaborative behavior, all of which are essential for leadership success.

Organizations tend to measure the things that are easy to measure. “The philosophy that dominates so many company cultures today is that initiatives that cannot be measured have no value. In most instances, that is a reasonable assumption. But it does not apply to leadership development — not, at least, in the quantifiable terms that dictate assessments of capital expenditures,” the authors of the MIT study conclude.

Pierre Gurdian and Thomas Halbeisen and Kevin Lane, writing in McKinsey Insights state: “when upward of 500 executives were asked to rank their top three human-capital priorities, leadership development was included as both a current and a future priority. Almost two-thirds of the respondents identified leadership development as their number-one concern. Only 7 percent of senior managers polled by a UK business school think that their companies develop global leaders effectively, and around 30 percent of US companies admit that they have failed to exploit their international business opportunities fully because they lack enough leaders with the right capabilities.”

They cite data which shows that for years, organizations have lavished time and money on improving the capabilities of managers and on nurturing new leaders. US companies alone spend almost $14 billion annually on leadership development. Colleges and universities offer hundreds of degree courses on leadership, and the cost of customized leadership-development offerings from a top business school can reach $150,000 a person.

The authors identify a number of reasons why development programs have been ineffective. Among them are:

• Leadership development programs try to do too much. “Focusing on context inevitably means equipping leaders with a small number of competencies (two to three) that will make a significant difference to performance. Instead, what we often find is a long list of leadership standards, a complex web of dozens of competencies, and corporate-values statements.”
• Move the learning from the classroom to the real world. “university-like settings) that offer participants time to step back and escape the pressing demands of a day job. On the other hand, even after very basic training sessions, adults typically retain just 10 percent of what they hear in classroom lectures, versus nearly two-thirds when they learn by doing. Furthermore, burgeoning leaders, no matter how talented, often struggle to transfer even their most powerful off-site experiences into changed behavior on the front line.”
• To change leader behavior, there must be a change in mindset. “Becoming a more effective leader often requires changing behavior. But although most companies recognize that this also means adjusting underlying mind-sets, too often these organizations are reluctant to address the root causes of why leaders act the way they do. Doing so can be uncomfortable for participants, program trainers, mentors, and bosses — but if there isn’t a significant degree of discomfort, the chances are that the behavior won’t change.”
• Failing to measure results. “We frequently find that companies pay lip service to the importance of developing leadership skills but have no evidence to quantify the value of their investment. When businesses fail to track and measure changes in leadership performance over time, they increase the odds that improvement initiatives won’t be taken seriously.

Too often, any evaluation of leadership development begins and ends with participant ; the danger here is that trainers learn to game the system and deliver a syllabus that is more pleasing than challenging to participants. Yet targets can be set and their achievement monitored. Just as in any business-performance program, once that assessment is complete, leaders can learn from successes and failures over time and make the necessary adjustments.”

According to David Rock, author of Your Brain at Work: Strategies for Overcoming Distraction, Regaining Focus, and Working Smarter All Day Long and T. Swart and colleagues, authors of Neuroscience for Leadership: Harnessing the Brain Gain Advantage our “executive brain” tends to function at its worst when we feel threatened, exhausted, bored or humiliated, and our executive brain functions at its best when we feel safe, are well rested, and we work on a specific tasks or goal that is meaningful, interesting or exciting.

David Rock argues from a neuroscience research perspective, most leadership development programs trigger a primitive brain response. Within most leadership development programs and performance appraisals, a leader’s personality and competencies are examined and judged by others according to specific standards and long lists of competencies.

Managers need to attend specific training and focus their attention on changing their behaviour according to the organization’s requirements. These trigger an automatic, unconscious threat response in their brain. Although they may rationally accept the feedback and agree that they indeed need to work on certain competencies or character traits, they unconsciously perceive the experience as a threat to who they are as an individual, to their autonomy in deciding how to run their own life, and their sense of safety in front of their assessor or superior (colliding with their need for Status, Autonomy, and Relatedness, according to Rock.

Michael Beer, Magnus Finnström, and Derek Schrader writing in the Harvard Business Review on why leadership training fails. The researchers noted problems with training programs as early as the 1950s, during the seminal Ohio State leadership studies. They found that one program had succeeded in changing frontline supervisors’ attitudes about how they should manage, but a follow-up study revealed that most supervisors had then regressed to their pre-training views. The only exceptions were those whose bosses practiced and believed in the new leadership style the program was designed to teach.

Then, in the 1980s, a study showed that training programs did not facilitate organizational change: Companies that tried to launch major transformations by training hundreds or thousands of employees across many units to behave differently lagged the only company (in a sample of six) that didn’t kick-start its transformation this way. The problem was that even well-trained and motivated employees could not apply their new knowledge and skills when they returned to their units, which were entrenched in established ways of doing things. In short, the individuals had less power to change the system surrounding them than that system had to shape them.

Those findings dovetail with research — by Amy Edmondson, of Harvard Business School, and Anita Woolley, of Carnegie Mellon — showing that organizations need “fertile soil” in place before the “seeds” of training interventions can grow. When the researchers looked at a corporate training program aimed at improving problem solving and communication between managers and subordinates, they discovered that success varied across the company. Improvements were greater in units that had already developed a “psychologically safe” climate in which subordinates felt free to speak up.

The researchers say that the mistake that those responsible for leadership development in organizations make is assuming that leadership development is an individual enterprise, rather than a collective one. By that logic, people must be selected for and developed with the “right” knowledge, skills, and attitudes in order to improve the institution’s effectiveness and performance. So HR defines the requisite individual competencies according to the company’s strategy and then sells top management on training programs designed to develop those competencies, believing that organizational change will follow.

But this ignores all the research on organizational development which sees organizations as systems of interacting elements: Roles, responsibilities, and relationships are defined by organizational structure, processes, leadership styles, people’s professional and cultural backgrounds, and HR policies and practices. And individual development doesn’t recognize that all those elements together drive organizational behavior and performance. If the system does not change, it will not support and sustain individual behavior change — indeed, it will set people up to fail.

Finally, HR expert Josh Bersin wrote an insightful article, “Why Leadership Development Feels Broken: And How We’re Fixing It,” says “The market for leadership development solutions is enormous: more than $14 billion is spent by corporations and there are more than 70,000 books and videos on the topic. When we ask companies about their top talent issues, “improving our pipeline of leadership” always comes out near the top. (As of the last year only 14% of companies have a strong bench of leaders, two-thirds believe a new leadership model is needed, 64% also believe their top challenge is developing “next generation” leaders.)”

Bersin goes on to say that while many companies still have hierarchies (they are being simplified rapidly, HPE just reduced its managerial levels from 62 to about 15), the hierarchy no longer defines how work gets done. People work in cross-functional teams, they work on projects, and their influence can be vast regardless of their level — especially if they are technical or customer specialists.

Bersin argues “ the idea that “leaders are more valuable than individuals” is over. Many companies now pay software engineers hundreds of thousands (to millions) of dollars if they’re superstars, and their managers may make much less. In fact in today’s world “everyone is a leader” at some point in their career, it’s just some of us that have “professional management” responsibilities as part of our job. We also have to understand that the model of “getting people ready for leadership” is also an old idea. Leadership, like HR, is a “craft” not a “profession.” In other words, you learn it by doing it, through coaching and apprenticeship, and by learning and reflecting on your mistakes. All of us who have led teams remember the time we hired the wrong person, treated someone in an effective way, or set goals or directions that didn’t go as planned. And we learned over time how to be better and better — always reflecting that ‘the right leadership approach’ varies depending on the situation, culture, and organization.”

Bersin concludes with this comment: “All this means that leadership development today is much more like what happens in the military: we have to ‘throw people into the water’ and “teach them to swim,” I believe the most effective companies now promote people into leadership before they’re ready, and then give them the tools and support to learn on the job, innovate with new ideas, and grow into their jobs in place. The idea of ‘waiting until someone is ready’ just doesn’t let the organization keep up.”

What To Do About It

My experience in more than three decades of experience as a CEO and senior executive coach convinces me that the focus on leadership development is in the wrong place. Most leadership development initiatives focus on competencies, skill development and techniques, which is some ways is like rearranging the deck chairs on a sinking ship. Good leaders need to become masters of themselves before they can attempt to be masters of anything else.

Leadership development initiatives need to focus on the following core areas to really make a difference in the type of leaders we have leading our organization:

• Prioritizing self-awareness and emotional self-mastery. Leadership development needs to be an inside-out process that focuses less on competencies and skill acquisition and more on increasing a leader’s self-awareness, and understanding how their behavior impacts others. Again, a superficial program of increasing emotional intelligence through techniques and tips of such things as listening skills, facilitation skills avoids or neglects the more important requirement of leaders to understand, manage and master their emotions and understand and respond appropriately to the emotions of others.
• Influence participants’ “being,” not just their “doing.” In soon-to-be-published research, Malcolm Higgs, Roger Bellis, and Deborah Rowland have found that leaders need to work on the quality of their inner game, or their capacity to tune into and regulate their emotional and mental states, before they can hope to develop their outer game, or what it is they need to actually do. So leadership development must start by working on the inner game. It’s very hard for leaders to have courageous conversations about unhelpful reality until they can regulate their anxiety about appearing unpopular and until they’ve built their systemic capacity to view disturbance as transformational, not dysfunctional. In order for leadership development to influence being-level capacities, the learning experience needs to offer stillness and space for intentional, non-obstructed contemplation. It’s difficult to teach how to be! Training people with tools and models is very different from simply holding a space for leaders to be. In practice, I have found that offering participants experiences such as mindfully walking outdoors in nature, sitting silently in peer groups to hear colleagues share their life stories, and providing out-of-the-ordinary tasks such as stone carving, enables leaders to tap into their inner world as a powerful instrument for cultivating the vital skills of purpose, self-awareness, empathy, and acute attentional discipline.
• Incorporate into training programs a deep understanding of the dynamics of human behavior on an individual basis. I continue to be amazed at how many senior executives have a thorough knowledge of skill areas such as finance or marketing, but lack a modern understanding of the significant neuroscience, human performance and psychological research that have enormous implications for managing employees, customers and vendor relationships. Also, executives often make the mistake of implementing behavioral based training programs on one-size fits all basis, ignoring individual personality and motivational differences.
• Require sustained long-term engagement. Leadership development initiatives are often a “one-time” effort organized around a book, a seminar or workshop, or a few coaching sessions, ignoring the fact that any substantial behavioral or attitudinal change requires continuity and a long-term commitment to be successful. Also, a focus on experiential learning, rather than cognitive and information driven skill acquisition is required.
• Implementing good training design. Often, leadership development programs, by default, are designed and implemented by and HR executive or department or by a senior executive, neither of which have had in-depth training themselves in the latest developments in neuroscience, psychology and human performance, nor have they themselves gone through an intensive personal development process of self-mastery.
• Require an individual personal stake in self-development. Leadership development is frequently seen as the responsibility of the organization rather than a shared responsibility with potential and current leaders. Rarely have I heard them say that they want to take personal responsibility to become the best person they can be and take charge of their personal growth.
• Develop an organizational culture that values individual and collective personal growth as a priority. Most organizations value things — such as financial returns, marketing, product development, etc.–over people development. And during difficult economic times, the people development and well-being budgets are the first to be cut.
• Choose people who know how to mentor, coach and develop leaders. The mistaken assumption is that individuals who are currently leaders in the organization or HR personnel by function are best suited to teach leadership, and coach and mentor leaders. This is like arguing that you are an educational expert if you went to school. Leadership development initiatives should be led by experts in teaching and learning and facilitation.
• Make a connection between leadership development and its application. It’s one thing to learn leadership theory and illustrative case studies. It’s another thing altogether to apply that to actual situations where the individuals are given an opportunity to grow and demonstrate their capacities;
• Incorporate mindfulness practices into leadership development. Good leaders are reflective and often introspective. The incorporation of mindfulness practices that emphasize those things would contribute greatly to a culture that balances “being” with “doing.
• Recruit potential leaders for development that are competent, humble, empathetic and compassionate and not driven by hubris or self-interest. Organizations continue to recruit leaders who fit the traditional stereotype of a charismatic, narcissistic male leader, who have little humility and a big ego.
• Leadership development should not be just for top performers. In an article in Harvard Business Review, Navio Kwok and Winny Shen argue “When it comes to leadership development, the business case for investing in the “best” given limited organizational resources appears straightforward: Individuals who have a demonstrated track record of success deserve to be recognized, right? They also seem like sure bets who will benefit the most from development opportunities because they have the requisite experience and capabilities to grow. But the individuals who receive the most development are often the ones who arguably need it the least. The authors call this the leadership development paradox. A by-product of the leadership development paradox is that the “rest” are typically excluded from those opportunities, creating disparities and perceived inequities within organizations, and it can ultimately lead to poorer work performance, decreased commitment to the organization, and greater intentions to quit. The authors present three common but potentially problematic assumptions that underlie the leadership development paradox and strategies for leaders to overcome those blind spots.”

Summary:

The solution to the problem of leadership development ineffectiveness lies in an integrated approach to the recruitment, shared responsibility for personal growth and the development of an organizational culture that sees leadership development as an organic process.

The idea that we could build a leadership pipeline through “development programs” and “a well-defined schedule of experiences” is no longer sufficient: we need to provide help through mentoring, coaching, and lots of external and internal exposure.

As far as coaching goes, the DDI E-Y research shows that mentorship (or coaching) is one of the most valuable tools we have. This study found that organizations with formal mentoring have 20% lower turnover, 46% higher leadership quality, and fill roles 23% more quickly. So while we do need development paths for people, it’s even more important to give them development coaches!

There has never been a more important time than now to recruit, select and train good leaders for our institutions and organizations. The welfare of employees, customers and our society depends on it. And we can start by discarding the old paradigms of what constitutes good leadership, and then redesign leadership development.