I have not spent an enormous amount of time in either Swedish youth hostels or gay brothels, nevertheless, when I say that the interior of (Fitz)Henry, a quietly funky bistro on the Leith docks in Edinburgh, looks like a cross between the two, I am reasonably confident of accuracy.

The ceiling is panelled in a curiously orange shade of varnished pine and the walls are hung with rococo flounces of heavy velvet or great sheets of thick linen, positioned just so. It is all rather odd, but in a nice way, which could easily serve as an advertising slogan for the place. The food is also a little odd. The service is certainly odd. But, as I say, mostly in a nice way.

I went with Ruaridh, one of Edinburgh's emerging literary lions, who used to live in a house in the Highlands with an eagle's cage attached, and who therefore seemed ideally suited to the Gothic surroundings. On a quiet Monday, we were waited on by the boss, Dave Ramsden - a thin, shaven-headed man with a gaze that penetrated through the gloom and a voice so deep that when he spoke the cutlery on our table rattled and the building's foundations shook. If the restaurant business ever lets him down, I'm sure he could make a healthy living voicing trailers for scary American movies.

For now, it is almost as if he is starring in one: he has a curious ability to appear at your side without you noticing - a neat trick to pull in a big room with only three occupied tables. And no sooner was he there than he was gone again, floating away into the shadows, leaving behind just the rumbling echo of that voice and the smell of sulphur. (OK, I made up the sulphur bit but the Gothic thing was all very odd - though, again, in a nice way.)

For his starter, Ruaridh ordered the chilled mussel and king prawn velouté with coconut milk and ginger, which struck me as a very Edinburgh thing to find on a menu. It was a dark November night, the wind outside blowing hard off the Water of Leith, but do they serve a hot soup? Not a bit of it. Hot soup is for southern nancies. In November, they serve theirs chilled. In December, it probably comes as a sorbet. I thought it a little on the underpowered side, but Ruaridh disagreed. It was, he said, a fine palate cleanser, and thick with good, fresh seafood. My steamed ballotine of rabbit confit with Savoy cabbage and haricot bean cream was the exact opposite: a solid, hearty and meaty dish. The perfect thing for a cold, early winter's night.

My main course was a genuine original: a whole baked red mullet, split down the gut and overflowing with a pine nut and squid stuffing, as though only in the very earliest stages of a brutal disembowelling. (The mood of the place is clearly getting to me.) It was an extraordinary piece of work, the fishiness of the squid playing brilliantly against the mullet. Ruaridh's main course of a stunning fillet of beef from the (relatively) nearby Buccleugh estate was exactly the kind of thing you want to find in an Edinburgh restaurant. There were a few other things on the plate - confit potatoes, a parsley emulsion - but they all played second fiddle to the marvellous piece of beef.

The puddings, it must be said, were not so much odd in a nice way as just plain odd. We passed on the chocolate and beetroot cake with orange sorbet and the frozen yogurt terrine with candied fennel and carrots. Instead I went for the warm egg doughnuts with vanilla citrus fruits, which read far better than it tasted. You wanted something straight out of the deep fat fryer but this seemed as if it had been hanging about on the plate a little too long.

Ruaridh's caramelised pine nut and sultana rice pudding looked pretty in its pastry shell, but was essentially just a great rice pudding that had been tragically interfered with. We consoled ourselves with a glass each of Tich dessert wine from a winery just across the river from Sauterne. It was rich and fragrant and nicely acidic. In April, the great vampire's castle that is (Fitz)Henry moves further into Edinburgh to a site which, appropriately, was once occupied by Scottish Widows. Somehow I suspect that the Swedish youth hostel varnished pine will not go with him, but doubtless the rest of the decor will. In any case, as long as Dave Ramsden and his voice are there to rattle the window frames the atmosphere will remain.

• (Fitz)Henry, 19 Shore Place, Leith, Edinburgh (0131 555 6625). Dinner for two, including wine and service, around £80. Contact Jay Rayner on jay.rayner@observer.co.uk.