Rumer, in a restaurant, is sensitive to events on other tables. The clang of dropped cutlery, a hand raised a little too long when calling for the bill, these are things that make the singer momentarily break from her own meal and eyeball the waiter, just make sure he's on the way. "I can't relax when someone needs the salt," she says, because for many years, before she had a hit record, back in the days when she was still known as Sarah Joyce, Rumer made her living as a waitress.

"I loved it," says the 33-year-old, dressed this lunchtime in a green-patterned dress over jeans, long, messy brown hair falling to reach the table. "The morning shift especially – pulling down the awning, setting out the chairs, putting on the music. Loved it."

We are eating in the lounge of the Union, a members' club in Soho: a curious, cluttered place with faded red walls and a general sense of its having an eccentric history. Rumer looks unimpressed when the young waiter fails to provide any background facts; a nugget or two of building trivia should, clearly, be a part of his professional arsenal. I bet Rumer was a good waitress.

"Depends who you ask," she says. "If you ask the chefs, they'd say I was awful. Chefs didn't like me because I tried to please the customers." She'd let people order off the menu, and then "have to dodge missiles when I got into the kitchen".

A good thing, really, that after 10 years of after-hours gigging, even working for a while as a popcorn vendor in a cinema, she got a deal with Atlantic Records two years ago. Her debut album, the mellow, 70s-influenced MOR of Seasons of My Soul, was a great success, selling more than a million copies. No more waiting on tables for you, I suggest, as our main courses arrive.

"I actually did a shift at my old restaurant a few months ago. Pullens, in south London. Does fantastic steak…"

Sorry?

"Pullens. It used to be a gentlemen's outfitters before it was a restaurant. The manager, Alan, used to be a fisherman so everything's …"

But I'm not after background facts. Rumer, a few months ago, would have been tweaking her follow-up to Seasons of My Soul, an album of surprising cover versions called Boys Don't Cry that at the time of our lunch is on its way to No 3 in the UK album chart. An odd time, you'd think, to start moonlighting at her local steak joint. What was she up to? "Just helping out," she says. "They were short-staffed. It was interesting – I was polishing cutlery and I thought, well, how many times have I done this? I used to stand doing the cutlery and dream about getting away, being a singer."

I wonder if her still relatively new career as a pop star, which recently included performing for President Obama at the White House, has been everything she expected. "Nothing ever is. There are times, now, when I wish I didn't have any of the pressure. All these experiences that I'm having – shaking Obama's hand, having a lovely chat with Michelle – they're amazing. But they're also laced with fear. Some people love performing but I don't have that confidence. Because I don't think that I'm amazing. I don't think everything that comes out of my mouth is perfectly executed."

Fame came late, at 30, and up until then Rumer's life had been disjointed: false starts with bands; bouts of depression; a slow year as a kitchen assistant at a hippie-ish retreat, cooking bean stews for visitors signed up for "gong camp, past-life regression, whatever". Now she is hurried around the world to sing and promote and glad-hand, "one minute Helsinki, the next New York". But self-confidence doesn't always develop as quickly as a career urged on by massive sales.

"Someone from my record label said to me: 'This bit about fame being hard. It's an interesting angle.'" Rumer hoots with laughter. "I said: 'An angle? It's not an angle, it's true. What are you talking about an angle ..."

I doubt, though, the label's angle-man got off with the affectionate chuckle she's using today. Rumer is, by most accounts, quite exacting when it comes to her music, and hands-on at every stage of her album's creation. Nothing irks her quite like her label telling her something can't be done.

"I ask them: 'Really? Or are you just a quitter?' I have the attitude that everything is doable. I've got a rogue gene. I've got these mountain goat herding genes from Pakistan."

These last comments need some explaining, if you're unfamiliar with the singer's past. Born to English parents in 1979, she grew up in an expat community in eastern Pakistan, where her father, an engineer, worked on the Tarbela Dam. They moved back to England to live in a house in the New Forest. "You know the expression 'fur coat no knickers'? That was us. We looked like we had money. Big house but it was falling apart. There was a swimming pool with no water in it."

At the age of 11, Rumer learned that she was the result of an affair her mother had in Pakistan. Her biological father was a man from the local village and the family cook. "We're talking about somebody who used to make my dad's jam roly-poly," she says. "Every night. And it was rock hard. He was the worst chef. Didn't know how to cook. Completely not interested in obeying rules. Did his own thing."

She bursts out laughing. "My mum told me once that he went shopping [for the family] and when he unpacked it he didn't know what to make of a box of Tampax. So he put it in the cupboard with the rice." Her mother and the man – who Rumer won't name – grew close, she says, because her mother was clever and frustrated. "They got on well because my mother was a linguist. The cook taught her Urdu, and the Qur'an. They studied together."

In her mid-20s, Rumer travelled to Pakistan to try to find her father, there to learn he'd recently passed away. Her mother had died from cancer only months prior. "I'd like to think I've buried all my sadness in this record," says Rumer, who worries she's spoken a little too often about her curious parentage. She says she has a tendency to "go dark" in interviews, especially when the subject comes up. "I haven't really got any skin left, you know?"

She says there's a third album in the works, a more cheerful one, but before she can finish it, "I just want to observe life again. Feel a part of the natural rhythm of life." With this in mind I ask if her brief return to restaurant work was an effort to recapture the calm before the clamour brought on by Seasons of My Soul. "I think I just like doing my own thing," she says. "I'm not very interested in being famous. It's good to make up for lost time, all the years I didn't earn money. But the fame … You can keep that. Not bothered."

Of course this is the sort of thing it's easy for an artist to say once they're successful. And if it came from a younger musician, you'd smile and nod and think: sure. Rumer, though, gives the impression of someone fairly honestly processing two bountiful but stressful years that followed a decade of meandering – polishing cutlery, cooking for hippies, handing out popcorn.

"There are good points and bad points to coming in late in the game," she says. "You adapt better when you're younger, you let it mould you. But I can't let it mould me because I already know what's real. I know what it's like to sit on the tube, going round and round on the Circle Line not wanting to get off and go to work. I know what it's like to wonder: where's my life going? To feel like you're getting older."

At this the waiter swoops in to take our plates, pleased to have a nugget of information to deliver. The Union opened in 1990, he tells us. Not a great fact, but Rumer checks the bill to ensure it includes a decent tip.

Boys Don't Cry is out now on Atlantic