Self-Organization: The Irresistible Future of Organizing
by Margaret J. Wheately and Myron Kellner-Rogers
Why do so many in organizations feel discouraged and fearful about the future? Why does despair only increase as the fads fly by, shorter in duration, more costly in their attempts? Why have the best efforts to create significant and enduring organizational change resulted in so many failures? Our organizations and we exist in a world of constant evolutionary activity. Why has change become so unnatural in human organizations?
We believe that the accumulating failures at organizational change can be traced to a fundamental but mistaken assumption that organizations are machines. Organizations-as-machines is a seventeenth century notion, from a time when scientists began to describe the universe as a great clock. Our modern belief in prediction and control originated in these clock-work images. Cause and effect were simple relationships; everything could be known; organizations and people could be engineered into efficient solutions. Three hundred years later, we still search for “tools and techniques” and “change levers;” we attempt to “drive” change through our organizations; we want to build solutions and to “reengineer” for peak efficiencies.
But why do we want an organization to behave like a machine? Machines have no intelligence; they follow instructions given to them. They only work in the specific conditions predicted by their engineers. Changes in the environment wreak havoc because they have no capacity to adapt.
These days, a different ideal for organizations is surfacing. We want organizations to be adaptive, flexible, self-renewing, resilient, learning, intelligent. These attributes are found only in living systems. The tension of out times is that we want out organizations to behave as living systems, but we only know how to treat them as machines.
Is it time to change the way we think about organizations. Organizations are living systems. All living systems have the capacity to self-organize to sustain themselves and move toward greater complexity and order as needed. They can respond intelligently to the need for change. They organize (and reorganize) themselves into adaptive patterns and structures without any externally imposed plan or direction.
Self-organizing systems have what all leaders crave: The capacity to respond continuously to change. In these systems, change is the organizing force, not a problematic intrusion. Structures and solutions are temporary. Resources and people come together to create new initiatives, to respond to new regulations, to shift the organization’s processes. Leaders emerge from the needs of the moment. There are far fewer levels of management. Experimentation is the norm. Local solutions predominate bur are kept local, not elevated to models for the whole organization. Involvement and participation constantly deepen. These organizations are experts at the process of change. They understand their organizations as a process of continuous organizing.
Self-organizing offers hope for a simpler and more effective way to accomplish work. It challenges the most fundamental assumptions about how organizations happen and the role of leaders. But it is not a new phenomenon. We have lived our entire lives in a self-organizing world. We watch self-organizations on TV in the first hours after any disaster. People and resources organize into co-ordinated, purposeful activity without planning. Leaders emerge and recede based on who it is available and who has information. Everything happens quickly and a little miraculously. These self-organized efforts create effective responses long before official relief agencies can even make it to the scene.
In the history of organizational theory, we have known about self-organization. Years ago, we called it “informal organization.” This was a description of what people did in order to accomplish their work. Often people ignored the formal structures, finding them ineffective and unresponsive. They reached out for the resources and relationship they needed; they followed leaders of their own choosing, those they knew they could rely on.
A more recent description of self-organizing is found in a new term that describes organizations as “communities of practice.” These “communities” are webs of connections woven by people to get their desire to accomplish. Xerox promotes based on their perception of needs and their desires to accomplish. Xerox promotes this concept, stating that a successful company must acknowledge the power of community, and adopt those “elegantly minimal processes” that allow communities to emerge.
And the World-wide Web is probably the most potent and visible example of a self-organizing network forming around interests, the availability of information and unbounded access to one another. It will be interesting to observe the Web’s future now that control issues have become a paramount concern of some.
While there are many other example of self-organizing occurring in our midst (including well-documented experience with self-managed teams), we will instead simply note that self-organization is not a new phenomenon. It has been difficult to observe only because we weren’t interested in observing it. As we describe organizations as a living system rather than as machines, self-organization becomes a primary concept, easily visible.
Order in complex systems.
In the natural science, the search to understand self-organization derives from a very large question. How does the life create greater order over time? Order is the unique ability of living systems to organize, reorganize and grow more complex. But the theoretical biologist Stuart Kaufman has demonstrated that the inevitable desire to organize is evident even in a non-living system of light bulbs. Kaufmann constructed a network of 200 light bulbs, connection one bulb to the behavior of only two others (using Boolean logic). For example; light bulb 23 could be instructed to go on if bulb 46 went on, and to go off if bulb 67 went on. The assigned connections were always random and limited to only two. Once the network was switched on, different configurations of on and off bulbs would illuminate. The number of possible on/off configurations is 10.30, a number of inconceivable possibilities. Given these numbers, we would expect chaos to rule. But it doesn’t. The system settles instantly (on the fourteenth iteration) into a pattern of on/off bulbs that in then continues to repeat.
A few simple connections are sufficient to generate orderly patterns. Complex behavior originates from simple rules of connection. Order is not predesigned or engineered from the outside. The system organizes itself. We live in a universe, states Kauffman, where we get “order for free”.
Emergence: The surprise of complexity.
Social insect, bird flocks, fish schools, human traffic jams, all exhibit well synchronized, highly ordered behaviors. Yet these sophisticated movements are not directed by any leader. Instead, a few rules focused at the local level lead to co-ordinated responses. Computer simulations that mimic flocking, swarming or schooling behaviors program in only two or three ruled for individuals to follow. There is never a rule about a leader or direction. The rules only focus on an individual’s behavior in relation to that of its neighbors. Synchronized behaviors emerge without orchestrated planning. (Recent commentators on the history of science note that scientists consistently avoided the conclusion that there was no leader. The belief in the need for planning and authority runs deep in western thought and its science.)
A startling example of complex and co-ordinated behavior emerging without leaders or plans is found in the species of termites that construct tower. In Africa and Australia, certain termites build intricate towers twenty to thirty feet high; these towers are engineering marvels, filled with intricate chambers, tunnels, arches, and air-conditioning and humidifying capabilities. Termites accomplish this feat by following a bizarre job description. They wander at will, bump up against one another, and react. They observe what others are doing and co-ordinate their own activities with that information. Without blueprints or engineers, their arches meet in the middle.
Whether it is light bulbs, birds, termites, or humans, the conditions that create organizations are the same. Individuals are similarly focused (termites know they’re building towers.) Members develop connections with one another. Each determines its behavior based on information about what its neighbors are doing and what the collective purpose is. From such simple conditions, working communities emerge. Self-organizing from local connections into global patterns and processes. Nothing is preplanned; patterns of behavior emerge that could not be predicted from observing individuals.
There is much to startle us in these scientific visions of how life organizes itself. Can human organizations be more intentionally self-organizing?
Three conditions of self-organizing
If complex systems emerge from simple initial conditions, then human organizations similarly can be rooted in simplicity. During the past few years, our own search has focused on the simple conditions, which support an organization’s capacity to access its intelligence and to change the needed. We have seen evidence of these conditions in a wide variety of settings: in world-wide manufacturers, in schools, in experiments with the future of battle strategy in the U.S. Army.
Organizations assume different forms, but they emerge from fundamentally similar conditions. A self gets organized. A world of shared meaning develops. Networks of relationship take form. Information is noticed, interpreted, transformed. From these simple dynamics emerge widely different expressions of organization.
These sense-making capacity of the organization efforts begin with intent, a belief that something more is possible now that this group is together. Organizing occurs around an identity-there is a “self” that gets organized. Once this identity is set in motion, it becomes the sense-making process of the organization. In deciding what to do, a system will refer back to its sense of self. We all interpret events and data according to who we think we are. We never simply “know” the world; we create worlds based on the meaning we invest in the information we choose to notice. Thus, everything we know is determined by who we think we are.
As we create perceptions of the world, we primarily use information that is already in us to make some sense of something new. Biologist Francisco Varela explains that more than 80% of the information we use to create visual perceptions of the world comes from information already inside the brain. Less that 20% of the information we use to create a perception is external to the brain. Information from the outside only perturbs a system; it never functions as objective instructions. Varela describes this in an important maxim: “You can never direct a living system. You can only disturb it.” This explains why organizations reject reports and data that others assume to be obvious and compelling. A system will be disturbed by information based on what’s going on inside of the organization, how the organization understand itself at that moment. This maxim also explains why organizations are never changes by assembling a new set of plans, by implementation directives or by organizational restructuring. You can never direct a living system; you can only disturb it.
The self-organization references include its vision, mission and values. But there is more. An organization’s identity includes current interpretations of its history, present decisions and activities, and its sense of its future. Identity is both what we want to believe is true and what our actions show to be true about ourselves.
Because identity is the sense-making capacity of the organization, every organizing effort – whether it is the start-up team, a community project, a nation – needs to begin with by exploring and clarifying the intention and desires of its members. Why are we doing this? What’s possible now that we’ve agreed to try this together? How does the purpose of this effort connect to my sense of new projects, teams, or community efforts? Did the group spend much time discussing the deeper and often murkier realms of purpose and commitment? Or did people just want to know what their role was so they could get out the meeting and get on with it? Did leaders spend more time on policies and procedures to coerce people into contribution rather that trying to engage their desire to contribute to a worthy purpose?
Most organizing efforts don’t begin with a commitment to creating a coherent sense of identity. Yet it is this clarity that frees people to contribute in creative and diverse ways. Clear alignment around principles and purposes allows for maximum autonomy. People use their shared sense of identity to organize their own unique contributions. (This critical partnering of high alignment and high autonomy also appears in Information Technology discussions as design criteria for creating effective distributed data processing or client server systems.)
Organizations lose an enormous organizing advantage when they fail to create a clear and coherent identity. In a chaotic world, organizational identity needs to be the most stable aspect of the endeavor. Structures and programs come and go, but and organization with a coherent center is able to sustain itself through turbulence because of its clarity about who it is. Organizations are coherent at their core move through the world with more confidence. Such clarity leads to expansionary behaviors; the organization expands to include those they had kept a bay-customers, suppliers, government regulators, and menu others.
Information: The medium of the organization.
Information lies in the heart of life. Life uses information to organize itself into material form. What is information? We like Gregory Bateson’s definition; “Information is a difference which makes a difference,” and Stafford Beer’s “Information is that which changes us.” When a system assigns meaning to data, “in-forms” it, then it becomes information. Complex living systems thrive in a zone of exquisitely sensitive information processing, on a constantly changing edge between stability and chaos that has been dubbed “the edge of chaos.” In this dynamic region, new information can enter, but the organization retains its identity. Contradicting most efforts to keep organizations at equilibrium, living systems seems to seek this far-from-equilibrium condition to stay alive. If a system has too much order, it atrophies and dies. Yet if it lives in chaos, it has no memory. Examples of both these behaviors abound in corporate America. The implosion of IBM and General Motors evidences how sophisticated information and measurement systems could create a sense of internal order while failing to allow for critical new information. And during the 1980s, many firms reached out chaotically without any sense of identity to markets and businesses that they were incapable of managing.
Information that flows openly though an organization often looks Chaordic. But the nutrient of self-organization. As one utility CEO aptly put it: ” In our organization, information has gone from being the currency of exchange, we traded it for power and status, to being the medium of our organization. We can’t live without it; everyone feeds off of it. It has to be everywhere in the organization to sustain us.”
Only when information belongs to everyone can people organize rapidly and effectively around shifts in customers, competitors, or environments. People need access to information that no one could predict they would want to know. They themselves didn’t know they needed it until that very moment.
To say that information belongs to everyone doesn’t mean that all decisions move to the most local units When information is available everywhere, different people see different things. Those with a more strategic focus will see opportunities that others can’t discern. Those on a production line similarly will pick up on information that others ignore. There is a need for many more eyes and ears, for many more members of the organization to in-form the available data to that effective self-organization can occur. But this is information – unplanned, uncontrolled, abundant, superfluous – that creates the conditions for the emergence of fast, well-integrated, effective responses.
Relationship: The pathways of organization.
Relationships are pathways to the intelligence of the system. Through relationships, information is created and transformed, the organization’s identity expands to include more stakeholders, and the enterprise becomes wiser. The more access people have to one another, the more possibilities there are. Without connections, nothing happens. Organizations held at equilibrium by well-designed organizations die. In self-organization systems, people need access to the intelligence of the whole system. Who is available, what do they know, and how can they reach each other? People needs opportunities to “bump up” against others in the system, making the unplanned connections that spawn new ventures or better integrated responses.
Where members of an organization have access to one another, the system expands, including more and more stakeholders. It is astonishing to se how many of the behaviors we fear in one another dissipate in the presence of good relationship. Customers engaged in finding a solution becomes less insistent on perfection or detailed up-front specifications. Colleagues linked by a work project become more tolerant of one another’s diverse lives. A community invited into a local chemical plant learns how a failure at the plant could create devastating environmental disasters, yet becomes more trusting.
The dynamics of self-organization:
These three domains of identity, information and relationship operate in a dynamic cycle, so intertwined that it becomes difficult to distinguish among them. New relationships connect more and more of the system, creating information that affects the organizations identity. Similarly, as information circulates freely it creates new business and propels people into new relationships. As the organization responds to new information and new relationships, its identity become clearer at the same times it changes.
Earlier we started that self-organization is not new in our experience of organizations it just takes different eyes to see it. Self-organization has been going on all the time, but our attention has been diverted to perfecting the controls and mechanisms that we thought were making work happen. It is our belief that most people, whatever their organization are using information relationship and identity to get work done. But the quality of what they have available is often quite poor. They work with whatever information is available, but it is usually insufficient. They need more; they create misinformation and rumors. But always they are organizing around information. People also work with whatever relationships the systems allows, often going around the system to make critical connections. Most people know which relationship would bolster their effectiveness, although this awareness may be voiced only as complaints. And as they do their work and make decisions, employees reference the organizational identity that they see and feel – the organization’s norms, unspoken expectations, the values that are rewarded.
When errors or problems occur, the real work is to look into the domains of self-organization and determine what’s going on at this subterranean level. In organizations, problems show up in behaviors, processes, or structures. Once we diagnose the problem, our collective practice has been to substitute new behaviors, new structures, and new processes for the problematic ones. But this seldom works. The problem that we see in organizations is artefacts of much deeper dynamics occurring in the domain of information, relationship or identity. If we can inquire at this deeper level, if we can inquire into the dynamic heart of organizing, both the problem and the solution will be discovered.
We observed the power of inquiring into these depths in a DuPont chemical plant in Belle, West Virginia. Safety had been a major focus for many years, addressed in many different ways. They had moved from 83 “recordable” injuries to zero. But after more than a year with no “recordable” injuries, three minor personal accidents occurred within a few months. The leadership team knew from past experience that the solution to their safety problems did not lie in new regulations. Instead, they examined the organization in terms of these originating dynamics of identity, information, and relationship. What were they trying to accomplish? Did they still believe in their principles? How were their relationship with one another? Did anyone still have access to all information? These leaders could have responded in more traditional ways. They could have initiated disciplinary action, more regulations, safety training classes, or increased supervision. Instead they questioned themselves more deeply and noted that because of several new members, they were no longer guided by the same-shared clarity about safety. The re-creation of that clarity restored them to superior levels of safety performance.
If self-organizing already exists in organizations – if people are naturally self-organizing – then the challenge for leaders is how to create the condition that more effectively supports this capacity. They do this by attending to what ever is available in the domains of information, relationship and identity.
Leaders in self-organizing organizations:
What do leaders do in self-organizing organizations? As their organization moves towards a mode of operating, which seems to exclude most traditional activities of planning and control, is there a role for a leader? Absolutely! Leaders are an essential requirement for the move to self-organization. This is not laissez-faire management disguised as new biology. Given existing hierarchies, only leaders can commit their organization to this path. But their focus shifts dramatically from what has occupied them in the past. In our work, we have observed many of the pleasures and perils of leaders on this path. We also are aware of some of the siren calls that seem to threaten the resolve of even the clearest of leaders.
The path of self-organization can never be known ahead of time. There are no prescribed stages or models. “The road is our footsteps, nothing else”, as the South American poet Machados wrote. Therefore leaders begin with a string intention, not a set of action plans. (Plans do emerge, but locally, from responses to needs and contingencies.) They also may have confidence in the organization’s intelligence. The future is unknown, but they believe in the system is talented enough to organize in whatever ways that future requires. This faith in the organization’s ability and intelligence will be sorely tested. When there are failures, pressures from the outside, or employee resistance, it is easy to retreat to more traditional structures and solutions. As one manager describes it: “When things aren’t going well, we’ve had to resist the temptation to fall back to the perceived safety of our old rigid structures. But we know that the growth, the creativity, the opening up, the energy improves only if we hold ourselves at the edge of chaos.”
The path of self-organization offers ample tests for leaders to discover how much they really trust their employees. Can employees make wise decisions, can they deal with sensitive information, and can they talk to the community or government regulators? Employees earn trust, but leaders create the circumstances in which such trust can be earned.
Because dependency runs so deep in most organizations, employees often have to be encouraged to exercise initiative and explore new areas of competence. Leaders have not only to let go and watch as employees figure on their own solutions, they also have to shore up their self-confidence and encourage them to do more. And leaders need to refrain from taking credit for their employee’s good work, not always an easy task.
While self-organization calls us to very different ideas and forms of organizing, how else can we create the adaptive, resilient, intelligent fast, and flexible organizations that we require? How else can we succeed in organization in the accelerating pace of our times except by realizing that organizations are living systems? This is not an easy shift-changing one’s model of the way the world organizes. It is work that will occupy most of us for the rest of our careers. But the future pulls toward these new understandings with an insistent and compelling call.