"Foods to be avoided: beefe, venison, hare, heavy wines, cabbage, fresh-water fish." With these words the 17th-century Oxford don Robert Burton outlined – in The Anatomy of Melancholy – his recipe for avoiding depression and achieving mental wellbeing. What you ate determined your happiness and soundness.

Hare was to be shunned with particular vigour, he argued. It is "hard of digestion, breedes incubus… and causeth feerful Dreames. So doth all Venison". By contrast, lean meats are best, as are "all manner of brothes, pottage, with borage, lettuce and such wholsome herbs are excellent good". For good measure, "egges are justified, butter and oyle may passe, while… sugar and hony are approved". Similarly, "the thinnest, whitest, smallest wine is best, not thicke, not strong".

Burton's book is a comprehensive analysis of the state of knowledge in the 17th century of the condition we now call depression and it assumes that diet is crucial to mental health. "Burton took it for granted, as did the rest of Britain, that a healthy body and a healthy mind went together," says Erin Sullivan, an expert on Burton, at the Shakespeare Institute, Stratford-upon-Avon. "The right meat and drink were therefore seen as being crucial to mental wellbeing."

Medicine has since made remarkable progress in revealing the origins of illness – with the possible exception of understanding how food might influence mood. Great claims are made for dietary supplements as depression treatments but they are controversial. As Paul Fletcher, a professor of health neurosciences at Cambridge University says: "Most claims made on behalf of isolated supplements and dietary components are unsubstantiated." The jury is out, in other words.

Nevertheless, many scientists remain convinced of a link. Dr Sadaf Farooqi, of the Cambridge Institute for Medical Research, believes research supports the idea that food can influence our moods and emotions. "Consider the naturally occurring enzyme monoamine oxidase which our bodies use to break down amino acids in our food," she says. Some individuals suffer from an inherited lack of that enzyme and so cannot properly metabolise products like cheese and red wine and have extreme reactions to themt. They have headaches and act aggressively.

"Now that is an extreme reaction, I admit," Farooqi adds. "However, levels of monoamine oxidase differ widely among people. These subtle variations could therefore contribute to predispositions to all sorts of behavioural and mood problems. So yes, food does affect mood and in different ways. It is just very difficult to pinpoint the mechanisms involved."

These points are backed by Professor Andy Smith, of Cardiff University. "Men and women certainly act as if they expect food to affect their behaviour – by consuming products that have virtually no nutritional value, such as alcohol or caffeine, because they know these are going to affect their moods."

Smith also believes foods affect mood though he is equally sure these effects are delicate and tricky to isolate. Certainly the field of nutrition and emotion is bedevilled with methodological problems. "Often, when we appear to have isolated a food that seems to trigger a change in mood, we find what is really going on is the reverse. Mood is affecting choice of food."

Smith looked at studies which suggested that individuals who sat down for carefully cooked breakfasts tended to have more positive outlooks on the day ahead compared with those who did not have breakfasts. From this, it was argued that a good meal sets you up for the day. The idea does not survive detailed scrutiny, however.

"We examined those people who had positive outlooks and found they had them whether or not they had breakfast. They were just that kind of person. They had a big breakfast because they were feeling good and were ready to get on with the day. Other people don't feel like that when they get up and there is no good trying to persuade them to sit down and have a leisurely meal."

Similarly, research on chocolate has shown that this also has an over-rated impact on the psyche. "There is some slight evidence that chocolate triggers the release of opiate-like chemicals in the brain but really its relationship with our emotions operates in the reverse direction," says Smith. "We seek out a chocolate snack when we feel upset or are emotional because, in the past, we have had pleasant associations with it. That is why it is a comfort food."

Professor David Benton at Swansea University split subjects into two sections and played "jolly" music (from Delibes's ballet Coppélia) to one while the other was subjected to Prokofiev's despair-inducing Russia under the Mongolian Yoke. "It was a toss-up between Prokofiev and Leonard Cohen," Benton admits. "Prokofiev only won narrowly."

Then the two groups were shown how to obtain chocolate drops by pressing a keyboard button on a computer linked to a dispenser. Those that just endured the Prokofiev clicked their computers almost three times more often than those that had listened to Coppélia, so desperate were the former to cheer themselves up. Mood clearly has a strong role in food choice.

To further complicate the picture, our beliefs about food can lead us to exaggerate any real nutritional impact they might have on our emotional state, adds Smith. "In one experiment, subjects were given a choice of cereal or a muffin to eat. Those taking the former believed they selected the healthier choice and their moods tended to be better afterwards. In fact, the researchers had picked muffins and cereals that were of almost identical nutritional value. The elevated moods of those who ate cereals were merely the result of their beliefs – or delusions, if you prefer – about what they had eaten."

US researchers who fed pieces of carrot to children found that the subjects rated those presented in McDonald's fries wrappers as being tastier than bits of carrot that were plainly wrapped. In reality, there was no difference.

Given these problems, establishing any kind of causative link between food and mood might appear near impossible. Nevertheless researchers have unearthed connections. Doris Stangl, at the Institute of Psychiatry, King's College London, does research on the adult hippocampaus, one of the few areas of the brain where new neurones continue to form throughout life, a process known as neurogenesis. The level of neurogenesis is closely linked to cognition and emotional states: essentially if new neurones are encouraged to grow, memory will improve while the likelihood of depression decreases. Crucially, Stangl's research has found that diet can affect the growth of neurones in the hippocampus.

"We found that intermittent fasting – eating fully one day and taking no food the next day – had a substantial impact on the growth of new neurones," she says. "In addition, reducing calorie intake by around 30% also boosted neural growth. In other words, meal frequency and calorie intact can affect neural growth. Now we are investigating meal content to see if that has an impact."

Stangl also points to studies that suggest that omega-3 fatty acids, found in high levels in fish, can alleviate depression. "I am convinced that the food we eat has an impact on our emotions," she adds. "I am not saying specific foods are going to be cures for depression but equally I do think that diet can have an impact on our moods."

Other research focusing on the neurotransmitter serotonin suggests this confidence is not misplaced. Low levels of serotonin in the brain are associated with depression. Indeed, most antidepressants act by raising serotonin levels. And this is where foods could play a role in mood regulation, some scientists argue.

They say that diets rich in carbohydrates will raise blood glucose levels and trigger a cascade of chemical changes that will increase levels of the amino acid tryptophan, a key component of serotonin, which would then raise amounts of the neurotransmitter, boosting mood.

"It's a convincing argument – though my studies suggest the amounts of carbohydrate needed to trigger a noticeable change in mood would be unrealistically high," added Benton.

"On the other hand, our studies do back the idea that carbohydrates improve mood. We asked 650 people what they had eaten that morning and found there was a distinct correlation between those who said they were happier and those who had eaten pasta or other carbohydrate-rich foods. I think there is a link though the mechanism involved is not yet clear."

It is not just what you eat that is important, says Ursula Werneke, a Swedish psychiatrist, it is the manner in which you consume it. "Meals give you a chance to stop and take a break from the stress of the day," she says. "More and more people eat at their desks and that is not healthy. You should take half an hour out of your schedule and relax. Grabbing a bite out of a paper bag is not going to do your mental health much good."

And similarly, when you get home, you should cut out the grazing, adds Werneke. "Have proper meals, relax and improve your general well-being – that is what food can do for your mental health."
Robin McKie is introducing A Feast to Cure Melancholy at Wellcome Collection, London NW1 on November 11 and 12