Project management is beset by an identity crisis. Like a teenager peering teary-eyed into a mirror and asking “Who am I?” it has struggled to define its position in the professional world. Almost everyone who works for a living has undertaken – and therefore “managed” – a project at one time or another, and yet we are not ready to give the discipline the credence it craves.

It suffers from its own ubiquity. In other words it is because, essentially, we are all project managers, (but we are not all architects, engineers, finance professionals and so on), that we don’t talk about it enough as a defined and ring-fenced discipline.

But where would we be without project managers, regardless of their zones of influence? Who would channel the creativity, stop budgets whirling out of control and make sure people at the top get what they ask for, not a pale facsimile of the stated plan?

Where would we be? Nowhere, that’s where. The wheel would never have made it into production, candles wouldn’t have been invented let alone light bulbs, and neither you nor I would be able to read this or any other article in this or any other newspaper, because none of it would exist.

Project management quietly rules the world. It greases the wheels of industry, keeps the lights on and the country moving. It creates transport infrastructure and utilities, enormous sporting events, such as the Olympics, and impressive works of architecture like The Shard in London, The Senedd in Wales and the Scottish Parliament Building.

But these examples, all completed in the last ten years, prove that projects rarely go to plan even in the modern era. Despite billions of pounds and thousands of jobs at stake, the industry suffers from inconsistencies at every level.

Project managers must keep a consistent dialogue with financiers, managers, contractors – even the general public

There is a dearth of people skilled in the practicalities of delivering projects and, as you read these lines, important jobs in the private and public sectors are going to individuals without the requisite experience to see them through.

It explains why some projects are applauded and others jeered. In the latter camp are unstructured and badly co-ordinated efforts, those in which the risks aren’t planned for and the benefits are not married to the objectives of the organisation that commissioned the work.

According to Alex Budzier, of the Centre for Major Programme Management at Saïd Business School, University of Oxford, too many organisations persist in taking piecemeal approaches and continue to make the same old mistakes.

“Projects never go according to plan,” he says. “However, managers can mitigate for risks and uncertainty. Our research shows that most projects struggle because their provisions were insufficient; a direct result of misunderstanding and misrepresenting risks.

“The reason often is the human tendency to imagine the future through the tinted glasses of optimism. Project leaders should de-bias future plans and consider what the last 20 projects aimed for and study the results. It gives more robust goals, more realistic timelines and the best estimate for the size of the risks.”

Another area of consistent failure lies in the realm of communication. Stakeholder perception of the project defines whether it is a success or not, so project managers must keep a consistent dialogue with financiers, managers, contractors – even the general public.

A rule of thumb here is the bigger the project, the more people are involved and the better the communication needs to be. Conversations and marketing materials should be targeted and relevant to each stakeholder, whether they are a student on work experience or the chief executive.

“The Shard, HS2 and Crossrail are all high-profile projects, with high public visibility, and as such you could consider each member of the public as a stakeholder with a different vision, sensibility or objectives,” says Sandrine Cuney, head of project management at Team Consulting.

“Therefore, it’s unlikely that general consensus will be met. This is likely to get worse in the future with the wide use of social media as a means of sharing information and opinions, fuelling discussions and influencing others.”

But she adds: “Prior to implementation of these projects, consultation at the outset with the targeted users is also crucial to confirm that the initial intent will meet the needs of the majority of end-customers or users.”

For years, groups such as Association of Project Managers and the Project Management Institute have been the flag bearers of best practice in this space, as well as the main firewall against slapdash approaches and hatchet jobs. But is the world finally waking up?

Just last year, the Cabinet Office announced the new Major Projects Leadership Academy (MPLA), delivered by Saïd Business School, from which senior civil servants must graduate before they are given the reigns of future public projects.

This and other initiatives are adding flesh to the bone of the skeletal project management sector and raising its status across UK industry. The sector’s self-doubt is gradually ebbing away and, as each new project that follows the proper course delivers the goods for its stakeholders, more and more bosses are catching on.