I’m in marketing,” a woman who looked startlingly like Catherine Tate’s office-worker sociopath character told me proudly on the Blue Train to Portsmouth Harbour. I hadn’t got her down as marketing director of Selfridges or Channel 4, but she could have worked in a promotional agency in Bournemouth at a pinch. As it turned out, she did data-inputting for one of the statistics warehouses that serve retail trade analysts. She could have done exactly the same job in McDonald’s offices or an NHS records department, but the tenuous link with the world of television commercials and rock festival sponsors clearly struck her as exciting.

I’m in marketing too. Sort of. Not that I say it quite like that, for all sorts of complex, pedantic and squeamishly English reasons. What I learnt to do in my first job, working for a high-table-talking Belgravia American in the 1970s, was clever market research – research a world away from the polls and statistics of commodity stuff. Research designed to shape strategies, not quantify the monthly round; the kind of research that was, in the words of one of my employer’s clever essays, “The bastard child of marketing and the behavioural sciences” – motivational research. Our reports were more fancy essays, involving intelligent taxonomies, elaborate conceits and a lot of art references. We wouldn’t have called it marketing then because that generic sailed too close to the cheery, cheesy world of promotions, direct mail and what advertising people called below-the-line; in other words, little better than sales.

When I set up in business with a partner, we described what we did as “research-based strategy”, aligning ourselves with the coming world of management consultancy. We chose not to align ourselves with advertising, the more obviously glamorous front end of marketing. Ad-land seemed a bit rackety. I’d learnt my MR trade before I wrote a word of journalism, but it came in handy, especially the taxonomy part – what we called “market segmentation” – when we described people, what they wanted out of life and how it explained their shoes.

If I wasn’t proud to identify with marketing it wasn’t just for those squeamish English reasons. Back then, thoughtful, liberal people told you that marketing was the devil’s work, an imported American practice and the thin end of a nasty wedge. A raft of middle-brow books – Vance Packard’s The Hidden Persuaders (1957) was the bestseller among them – had described marketing, and especially advertising, as manipulative and dishonest, the advance guard of dumbed-down materialism. Later, of course, the world changed utterly and marketing started to sound cleverer and more directional.

This week, hundreds of marketers – the marketing business’s word for its practitioners – made their way to the first ever Edinburgh International Marketing Festival, held in venues already brimming with cultural creatives and comedians and talent-spotters. Had the marketer from Bournemouth attended her confidence would have been vindicated. The organisers are the marketing establishment – The Institute of Practitioners in Advertising (IPA) and the Marketing Society – in league with a new-generation business called Creative Brief. The audience, of people with titles like marketing director, brand manager, digital comms director, consumer insight wallah, clearly reckoned that if they got away from the sort of workaday conference held in purpose-built hotels in Hounslow (porridge-coloured composite marble floors, velveteen chairs) to somewhere altogether fringeier, funkier and younger – the main venue was the city’s Assembly Rooms, incubator of all those famous stand-up careers – they’d be able to debate what they’re about more freely.

This is a difficult moment for marketers. Market Leader, the Marketing Society’s clever, literate journal, has been debating in recent issues whether the internet has changed absolutely everything in their world, or just most of it. Judie Lannon, its editor and a former planner at J Walter Thompson, says: “Marketers get distracted by the accountability pressures, all the New Age agencies that report to them and all the online one-hit wonders they have to understand.”

The festival programme was packed with debates and talks about the internet, about social responsibility, about the environment and the future of the human race; and what shall we do about the recession? The combination of the credit crunch and the internet has been disastrous for a range of traditional advertiser-supported media – particularly newspapers and television, the pillars of the postwar marketing landscape. Until recently, big league marketers worked on the assumption that whatever else they did, they could reach mass audiences predictably and regularly through a stable media universe of print and electronics, newspapers and magazines, radio and television. They knew the environments for their messages, the precise way they would be delivered, the profile of their audience, because it had all been researched to death.

In the 1980s, for a blissful decade or so, marketing seemed set to run the show – the government, UK plc, the lot. Saatchi & Saatchi was a fast-growing FTSE100 company; British television advertising, the most familiar face of marketing, was on a roll, popular with the (still huge) audiences for terrestrial television, and admired internationally. Headhunters were even saying that with a bit of training and conventional tailoring, the top marketers could form the next cohort of chief executives. But all those certainties have gone, replaced by a more confusing world of fragmented, “long tail” markets and demanding, disloyal, “brand me” consumers.

The problem for capital-M marketing people is that we’re all marketers now. The language, the ideas, the mindset of marketing have become part of everyday discourse. These days, if the marketers aren’t on the board themselves, people with what the bestselling US business writer and Harvard professor Theodore Levitt called the marketing imagination are. Back in 1982 Levitt’s The Marketing Imagination defined a way of looking at the world and doing business in it: an instinctive belief that the route to growth and profit lay in looking outside the business, understanding and satisfying customers, producing what they wanted, building brands (“beach-heads in the mind”) that would command premiums (higher prices) and loyalty.

The people who told me to speak English back then when I came out with my marketing-speak patter – the would-be literary novelists, contemporary artists or radical barristers – all use it now. They describe things as “up-” or “downmarket”; they talk about “target markets” and “branding” of charities, theatres and countries. They discuss “segmentation” and “demographics”, “profile” and “penetration”. Above all, they go on about “getting some PR” – often for themselves.

The great irony – the pyrrhic victory – of marketing is that while it’s having its moment of doubt, its ideas and language seem practically to have conquered the world. The core intellectual precepts of marketing, such as customer focus (being dedicated to customers for life) and brand-building (making the brand central to everything the company does) and consultation (constantly sounding out how people in your market think, feel and behave and feeding it back into everything you do), have become part of life not just in business but in the public sector and the non-profit world of institutions and charities, as well as in personal life.

Since the 1980s, the public and voluntary sectors have adopted marketing practices. They, too, became customer- and client-facing, they researched and PR-ed and sought partnerships with big-brand sponsors. And the bigger ones advertised, too, to raise awareness and funds.

In the 1990s, departments of state and good works quangos developed something called “social marketing”, using the toolkit of marketing to achieve good behavioural changes – all those Don’ts and Stops, for everything from drunk-driving to taking drugs to eating too much. Every big-spending government department has marketing-literate people in it now, often high-fliers hired from the private sector. And there’s a covert consensus in the political centre ground that they’re as much part of the modern world as being digital; they’re necessary, uncontroversial – even democratic. Greater Good: How Good Marketing Makes For Better Democracy (2008), co-written by John Quelch, a Harvard Business School professor, represents the zenith of this sort of argument. And although the Coalition has cut government advertising by half, it has also set up a Downing Street “nudge” unit, the Behavioural Insight Team.

In the 21st century, marketing has come home to individuals. The move to a “Me Inc” approach of “personal branding”, the growing “how do I look?” obsession, massaging your presentation to the world through your Google entry and social media presence, your blogs and tweets and personal websites, all derive from marketing. It’s what Norman Mailer referred to ironically as “Advertisements For Myself” in his 1959 collection of show-off essays of that name, but it’s not ironic now, for literary intellectuals or anyone else.

Of course marketing has more to be proud of than its total penetration of establishment thinking and language. There’s always been a close relationship, bordering on the symbiotic, between marketing and the arts. The exchange between the two is as apparent in Millais’ Pears Soap “Bubbles” and the bottle of Bass in Manet’s bar at the Folies-Bergères in the 1880s as the moment in the 1960s when advertising seemed practically to have merged with contemporary art: pop art took themes from the ad world, product packaging and commercial pop-culture, and smart advertising took them back. Advertising creatives were forever loping around galleries, goofing off to experimental film festivals, listening to new music, relentlessly talent-spotting, looking for the tricks and the tropes that would differentiate their clients’ brands.

In his The Conquest of Cool (1997), the left-leaning US political commentator Thomas Frank says that Madison Avenue practically invented the 1960s cultural youthquake, rather than just buying it out. Of course a cohort of writers and film directors, from Fay Weldon and Salman Rushdie to David Puttnam, Alan Parker, Ridley Scott and Tony Kaye, started out in advertising.

Today, beset by the new media landscape and post-crunch capitalism, the people who gave us what we wanted are having one of their funny turns. The period gorgeousness of TV’s Mad Men world, the kitschy interest in the plastic roses Daz gave away in the 1960s, the big sweep of big brands, big ideas and big budgets, can all look so dangerously retro, fragmented by the 2010 agenda.

But marketing men and women need to keep their nerve, raise their eyes back up to the hills; if we’re entering a brave new world of marketing, it’s one obviously saturated by the language of the industry. It’s a world where the quintessential scenario of 21st-century self-branding, the digital equivalent of the old 30-second television commercials, is your very own YouTube series (“My Gorgeous Life and Friends”, “My Struggle with Braces”, “My Diet” – all chances to talk about Me).

Marketers, you’ve won. It may not look exactly like that as you work out whether the anarchic-looking world of viral marketing, social media and YouTube can possibly yield anything like the neat schedules and steady returns of the golden age. But you have to remember that your company, your institution and your country really need you.

When discredited financial engineering and the last decade’s gunboat diplomacy have run their course, when the exciting new technologies have begun to sour, those people who can listen well and talk compellingly inside and outside the organisation – whatever it does – those who believe in organic business growth, can take over once again and do what the Harvard international relations specialist Joseph Nye famously calls “soft power” on the side of the angels.

Peter York is chairman of SRU