Victoria Park. 1990. Game day. Union leader Charles Livingstone is watching a young ground attendant, "decked out in a dustcoat and looking out of place". This is the home of the Collingwood Magpies, and the epicentre of Melbourne's old Labor heartland.

Livingstone is facing election as the secretary of the theatrical employees' union, representing workers at sports grounds, cinemas and theatres. He's heard murmuring of a challenge from a group of young Labor right-wingers led by a private school boy named Shorten. "I was on the hustings and I came across this guy on a landing on the old Sherrin Stand. He took my material, harrumphed at me, and wandered off. He was stand-offish, and when I heard his plummy voice I knew it was Bill – masquerading as a ground attendant."

Opposition leader Bill Shorten during question time this month. Photo: Alex Ellinghausen

Bill Shorten is one of Australian politics' best known but least defined characters; a shape-shifter. His vast network spreads from the heights of business boardrooms to the drab backrooms of the labour movement. From the radical right to Maoist left. He has used rare political cunning over 25 years to build, patch up and rebuild an often fragile power base in the ALP. A media master, he has been meticulous in smoothing bumps in his personal story.

Yet, after 20 years in public life, major questions hang over the alternative prime minister. Who is Bill Shorten and what does he stand for? Everyone's mate, when it suits. Charismatic, at times cantankerous, always calculating. Close up he is almost Clintonesque in his charm, a man in constant motion, adding and culling alliances, friendships and policy positions.

Bill Shorten with his wife Chloe. Photo: Simon Schluter

Here he is at Tasmania's Beaconsfield mine advocating on behalf of miners trapped beneath ground. Here he is hob-nobbing at Kew and Toorak mansions. Here, again, espousing open markets. And, now, at a union picket railing against Japanese imports.

Perhaps it is a measure of the man's personal and political breadth, his ability to encapsulate seemingly contradictory stances. Yet it is this very changeability that unsettles even those close to him. A sense that everything is secondary. Even footy. That the flight from a failing team in exile to the most supported, most talked about team in the country, may have nothing to do with football at all.

"He changed from South Melbourne to Collingwood late," says the Labor Right mate over his coffee cup. "I don't know why."

***

When Charles Livingstone first encountered Bill Shorten at Victoria Park, the younger man, then 23, was also working as an adviser to a state Labor minister, studying arts/law at Monash, and running the Young Labor Network (the Networkers) – an ambitious and privileged bunch of young ALP moderates including future QCs, union leaders and state ministers. Even then, Shorten stood out.

"He was the most amazing person I had ever met," says Aaron Patrick, a friend at the time, fellow Networker, now journalist. "He was intelligent and funny and confident."

Later, long after he had fallen out with Patrick and others in the group, Shorten, in a 2009 Fairfax profile, would dismiss the shift in footy allegiance as an inevitable consequence of South Melbourne's move north to become the Sydney Swans in 1982. In a later profile Shorten also gives the impression that his Victoria Park job was a weekend gig watching his beloved Pies – and that from there he and workmates had launched a challenge to the moribund union they had joined. But like much of his wider story, the truth is not quite as Shorten would have us believe.

In fact Shorten and his Network mates had organised jobs at sports grounds, race tracks and cinemas as part of a deliberate plot to take over Livingstone's left-wing union. "Lots of us took jobs as ushers and other things," recalls another old Networker George Habib, now a psychologist, and no longer involved in Labor politics.

As it turned out, Shorten's first takeover bid ended badly, the cashed-up Networkers misreading the mood of the union's rank and file. "Our organisers spent a lot of time at the grounds working with members," recalls Livingstone. "The demographic of the members was overwhelmingly older. So, a bunch of uni students with no work experience had no chance – it was a shellacking."

But the experience had revealed Bill Shorten's preparedness to immerse himself for his cause, even when uncomfortable.To don a guise where necessary. To work with disparate people and within groups as required.

Bill Shorten (centre) in the Xavier Senior Soccer team in 1979.

In a way it was what he was born for. Shorten's mother Ann was a teacher and, later, academic and lawyer who sent her twin boys Bill and Robert to the elite Jesuit school Xavier College from a young age, and instilled in them the value of education. His father Bill Snr worked as a manager on the Melbourne docks, exposing Bill junior (if from a distance) to the gritty romance of Melbourne's old working class – a working-class credential he has been at pains to emphasise throughout his political ascendancy.

It was an upbringing that prepared him for a professional and personal life moving seamlessly between polarities. The Xavier boy who went on to head the AWU. The son of a former fitter and turner who went on to marry the daughter of a Liberal MP. His defining characteristic may be his ability to bridge gaps that for many seem unbridgeable.

At his best, he is impressive. Australians came to know Shorten in arguably his finest hour – as the public face of the efforts to rescue trapped miners after an earth tremor triggered the Beaconsfield mine collapse in Tasmania in 2006. He spent almost two weeks updating the media about the underground rescue effort, winning the respect of miners, their families and mine management.

It was the ultimate expression of his ability and willingness to identify and seize opportunities to merge his own heartfelt acts with political pragmatism. It was the pinnacle of his years as a union leader; It was also a sublime segue to a parliamentary career. Later, in government, his determined push for action on disability insurance was both effective and celebrated in a similar way.

Shorten's handling of the mine disaster was reminiscent of the man who also reminded some seasoned Labor observers of his closest role model, former Labor PM Bob Hawke.

Shorten leads a group of miners into an AWU meeting in Beaconsfield in 2006. Photo: Janie Barrett

***

A Carlton restaurant. Late 1980s. Prime Minister Hawke arrives to speak to a group of Labor Right activists barely in their 20s. Greeting him at the white Commonwealth car is a young Bill Shorten. Those present are struck by the respect shown the young man by the older. "Hawke knew who Bill was," says George Habib. "Bill was already accomplished, in a very Hawke way." Despite the age difference, the two had much in common including charisma and big egos. (Monash University student magazine Lot's Wife carried a regular Dirt Column including a Hack's Top Thirty hits dedicated to the political players on campus. At number 3 on 1987 was Carly Simon's You're So Vain, dedicated to Bill Shorten.)

Both also had reputations as ladies' men. Each would also go on to divorce their first wife – Shorten leaving Deb Beale, the daughter of wealthy former Liberal MP Julian Beale for Chloe Bryce, daughter of former governor-general Dame Quentin Bryce.

Bill Shorten and his former wife Debbie Beale in 2007. Photo: Simon Schluter

Comparisons are often also made about the two leaders' political philosophies. Hawke arrived on the political scene precisely as the long post-war economic boom was coming to an end, and with it the welfare state that had underpinned Old Labor. New Labor was on its way.  Hawke duly morphed from beer swilling everyman, union boss and nationalist to sober champion of internationalism, open markets and cooperation with business. In doing so he established the prototype of the new Labor model that others like Britain's Tony Blair would emulate.

Like Hawke, Shorten has sought to walk the tightrope between the disparate worlds of business and labour, bosses and workers, rich and poor.

In 1994, after a stint at labour law firm Maurice Blackburn, the rising Labor star became an organiser at the AWU, the oldest union in the country.

He shone as an organiser, overcoming class suspicion – and a soft boyish look – to win the confidence of many members in factories, building sites and shearing sheds. Union colleagues from that period recall old-fashioned stump speeches by the young official that would transfix his worker audience. Within six years he was both national and state secretary of the rejuvenated AWU.

Bill Shorten when he was boss of the AWU, pictured in 2002. Photo: James Davies

At the same time he cultivated relationships with employers. He was especially close to late industrialist Richard Pratt, a family friend of first wife, Deb Beale's family. Pratt came to treat Shorten like family.

In the early 2000s even Kerry Packer, then Australia's richest man, lent a hand to a Shorten idea by agreeing to revamp the union magazine, The Australian Worker, as a "lifestyle" mag, and to sell it on newstands.

Meanwhile, as his political network continued to widen, industry moguls and steel mill delegates would happily turn up at Crown Casino for the annual AWU balls that Shorten founded. Like Hawke, Shorten seemed more than comfortable amid the trappings of wealth.

Shorten counts among his closest friends a disparate group, among them fellow Xavier old boy and head of the hardline free-market Institute of Public Affairs, John Roskam, and notorious blogger and sometime business consultant Andrew Landeryou. Another friend was and is Ray Horsburgh, now chairman of transport giant Toll Holdings, and former head of Smorgon Steel (now One Steel). "I found he [Shorten] aligned what he was trying to achieve in representing workers with what management was trying to achieve for the betterment of the business," Horsburgh says, not for the first time.

Yet if Shorten inherited modern Labor, he also personifies the tensions that remain in it, both professionally and personally.

In the early 2000s he fostered a relationship with militant manufacturing union leader Craig Johnston, who was facing charges and a jail sentence for his part in rampages through employers' offices in 2001.

Shorten can inspire fierce loyalty. He still has friends from his picket-line days. Joan Doyle was a long-time postie and militant union activist and now secretary of the postal workers' union. She recalls Shorten on picket lines and does not doubt his sincerity. "I think he understands working people," she says.

During question time this month. Photo: Alex Ellinghausen

In recent years Shorten has been close to Victorian plumbers' union secretary Earl Setches, a bonding entrenched in 2008 when the then left-wing knockabout hiked the Kokoda Track with Shorten, and came back reborn as a right-wing Shorten loyalist.

But other union leaders are not sure which side Shorten is on, the boss's or the worker's.

"The best way to characterise it is a love/hate relationship. I have no idea when he's going to love me or hate me," says airline engineers' leader Steve Purvinas, a rare modern-day unionist to rise from the shop floor. "He hates me when, as government minister, he screws workers by supporting Qantas when it grounds its fleet. Other times he and his party give me fantastic support."

There is another, less happy, side to Shorten's friendships. A history of strategic alliances, friends wooed and then apparently dropped. A growing list of unhappy exes.

***

Dean Mighell's steely blue eyes grow distant as he shakes his head and fiddles with the cup in front of him. It is a rare moment of self-doubt for the left-wing union firebrand and former Victorian leader of the electricians' union. It is a particular look common among political players drawn into the orbit of Bill Shorten and set adrift; a look of someone caught up in a game they do not fully understand.

Mighell is recalling the mid-2000s, ahead of a bitter round of federal pre-selections in Victoria, including for the safe seat of Maribyrnong in Melbourne's west, coveted by Shorten.

Mighell was a key figure in an alliance between Shorten's moderate AWU and hard left-wing unions including the wharfies, posties and plumbers. With his own faction split, Shorten would rely on the alliance for all-important votes for his own pre-selection.

Mighell and Shorten spent time together as friends – one of Bill's "new loves", as Shorten's first wife Deb Beale was known to describe her husband's many political romances. In mid-2000s Shorten was almost inseparable from Kathy Jackson, the Health Services Union leader who would go on to become a whistleblower on union corruption before facing allegations of similar charges herself. That relationship would end in a bitter public spat, Jackson accusing Shorten of dodgy factional deals and dubbing him the "Prince of Darkness".

Jackson is one of the few slighted friends to publicly hit out at Shorten. Former Networker Aaron Patrick is another: "I loved the guy," Patrick told Fairfax Media last year. "But then, when I was no longer of use politically, he ended the friendship and we didn't have any contact after that."

Shorten, as AWU state secretary, speaking to workers in 1999. Photo: Craig Sillitoe

Mighell is more philosophical. But he acknowledges his friendship with Shorten came at personal cost. Bob Sercombe, whose seat Shorten was intent on taking, was a friend of Mighell's and his local MP. "Knocking off Bob Sercombe was personally the hardest thing I did. I liked Bob. But Bill had earned some credibility in our ranks for his work as a unionist, and our policy was to support trade unionists [for parliament]."

Once Shorten secured his seat, says Mighell, the calls stopped coming. The new member for Maribyrnong had moved on. Mighell does not regret supporting his former union buddy but concedes he, and the electricians' union, may have been used for their votes in the ALP. "One thing about Bill Shorten is that if you're no value to him, he doesn't go out of his way to be your friend."

It is unsurprising, then, that even some who regard themselves as friends wonder what Shorten really stands for.

So which one is the real Bill? Modern Labor man or class warrior? As Opposition Leader the answer is no clearer than it was 10 years ago. Since moving to Canberra Shorten has struggled to reconnect with his Beaconsfield persona. On his feet he appears uncertain, as if choosing from a palette of Bills. Like the social butterfly who has finally got all his friends to a party and is lost in the existential dilemma of who to be.

Business leaders were taken aback by a truck tray address at a union rally at Australian Submarine Corp in Adelaide last year in which Shorten railed against the proposal that Australia's submarine contract might go to Japan – a protectionist stance at odds with New Labor. The Adelaide speech had friends and foe alike scratching their heads and asking yet again, "what is Bill really about?"

Former plumbers' union and AWU official Nathan Murphy belongs to Shorten's sub-faction within the Victorian ALP. He is not alone in his concerns about his leader's policy shifting and public profile. "I think people have an issue with Bill. His approval ratings are pretty low for an Opposition Leader against an unpopular Prime Minister. I think people can't get a hold on what he stands for."

After more than than a year in the spotlight as Opposition Leader, Shorten was stumped when the ABC's Jon Faine asked what he stood for. "Everybody is somebody," he responded, clumsily citing Dr Martin Luther King.

Childhood friend John Roskam makes a point of never publicly criticising his old mate and marital best man. But asked about Shorten's economic policies, Roskam responds by talking about the party he leads. "Is it the modern Labor of the Hawke-Keating years or old Labor of the pre-Whitlam era?," he asks. "Bill as leader knows what needs to be done. He has to push out the boat and have people follow."

From an altogether different vantage point, World Vision head Tim Costello agrees. He has worked with Shorten on policy matters over a number of years, but says that as Opposition Leader Shorten is  politically unconvincing . "I fear that the early signs show a person who is still sniffing the wind rather than having an unequivocal, clear vision for the nation."

***

A warm evening. Late 2009. Bill Shorten and his new wife Chloe are in the backyard of their Moonee Ponds home with select guests. It's a reception after their wedding in St Thomas' Anglican Church in Melbourne's inner west.

Newlyweds Chloe Bryce and Bill Shorten in 2009. 

To the surprise of many, including fellow MP and Catholic David Feeney, Shorten at age 42 has converted from Roman Catholicism to Anglicanism ahead of the event.  Feeney, who has been in a 25-year on-again-off-again political romance with Shorten, rounds on his mate and mocks Anglicans for the lack of enthusiasm in their faith.

It is all in good humour, sort of. But the conversation is not really, or not only, about religion. It's about their host's willingness to remake himself, shift allegiances, policies, friends – now even religions – as it suits. The reality is that while Australia's alternative prime minister identifies as Christian, he is more pragmatist than puritan. And anyway, football is the real religion in Victoria.

The swap of teams, by the way, did not coincide with South's move to Sydney. Shorten changed  in the late 1980s, seven years or so after the Swans did, when he was at least 21, probably 22, and became a Magpies supporter.

"I think there was an eye on the bigger prize with that decision," says one old comrade from the time, also recalling that Shorten was not particularly interested in the footy.

Another old friend, George Habib, also remembers it well. A Pies supporter, Habib recalls encouraging Shorten.

"I do remember on numerous occasions saying, 'Mate, if you want to become Prime Minister of this country you can't stay with the Sydney Swans.'"