Victor Hugo​ was excessive, in life as in literature. Cocteau said that ‘Victor Hugo was a madman who thought he was Victor Hugo.’ The critic and gardener Alphonse Karr wondered: ‘What was the point of going to all the trouble of becoming Victor Hugo?’ His English biographer, Graham Robb, wrote that Hugo’s life was ‘an inspiring lesson in the art of surviving one’s own personality’. Hugo even entered the Guinness Book of Records for having written (in Les Misérables) the longest sentence in literature before Proust came along. When Hugo indulged in table-turning, only the greatest emerged from the shadows: Dante, Napoleon, Socrates and Mozart; Hannibal spoke to him in Latin, while Shakespeare dictated a whole comedy, usefully in French. His vanity was often preposterous. In 1873 the 71-year-old genius was busy seducing his new maid, Blanche; when she touched his penis, he explained to her: ‘It’s a lyre … and only poets know how to play them.’

It’s estimated that Hugo made 4000 drawings, of which about 3000 survive; 77 are shown in Astonishing Things: The Drawings of Victor Hugo (until 29 June). This is one of the brownest exhibitions I’ve ever seen: brown ink, brown wash, brown pencil, occasionally lightened by black. Even when Hugo’s own work is interrupted by the dozen albumen prints of Hauteville House (his place of exile on Guernsey), the colour remains entirely consonant with the rest of the show. Which means that when, occasionally, Hugo decides to add a patch of colour, it blazes out with twice its normal effect. Meeting Room of the Municipal Council of Thionville, after the Entry of the Prussians shows the interior of a wrecked building. Through the blown-out windows can be seen a sky which is not the usual brown but actually light blue. Somehow this registers not so much as authentic but as shockingly original. The wall labels announce the conventional media – charcoal, graphite, ink, gouache, crayon and gum – which Hugo used. But there were also some more unconventional ones, what Robb calls ‘a whole pantry of other substances: blackberry juice, caramelised onion, burned paper, soot from the lamp and toothpaste … coffee grounds … food stains and smut’. (The ‘smut’ serves to cover rumours that Hugo also made artistic use of certain bodily leakages.)

Henry James thought the weakness of Roman civilisation was that it was only good at doing large things; he judged the Pont du Gard stupid, adding: ‘The Roman rigidity was apt to overshoot the mark, and I suppose a race which could do nothing small is as defective as a race that can do nothing great.’ One of the joys of this show is that Hugo, author of vast fictions and prolonged poems, was also very good at doing small, even very small, things. He can concentrate a whole abandoned park into a space four and a half by three and a half centimetres. Lace and Spectres (c.1855-56) is almost as tiny, but its two Japanese-style death’s heads (or perhaps the same head shown in full face and profile) roar out at you like the biggest fiends from Hell. The show engulfs you in Hugo, and very rarely reminds you of other artists. Ink-blackened Page with Half-moon and Fingerprints, a view upwards from the interior of a well with human heads (made up of the artist’s fingerprints) peering down, echoes Goya. But the only time I felt the strong presence of another artist, it was not imitative but proleptic. Planet-Eye (c.1854), a vast eyeball floating in cosmic clouds, sharply anticipates Odilon Redon’s Oeil-Ballon by 25 years. Redon’s noirs were to inhabit a parallel psychic zone, full of eyes floating in monochrome skies, grinning spiders and crackly skeletons.

These drawings are the negative image of Hugo’s vastly populated fiction. Here human life is barely in evidence. We are presented instead with landscapes which are not topographical, seascapes where there are no calm waters, with ruined buildings, castles both fantastical and destroyed, plus menacing cliffs, towering clouds, bleak fortresses, dead cities and blasted heaths. So, as with those sparse irruptions of colour, when we come across a drawing called The Cheerful Castle it feels like a blasphemy against normality. And it is a relief to get back to The Shade of the Manchineel Tree, that infamous West Indian growth whose sap and fruit are poisonous, and whose very shade is toxic. This is a world in which terrifying serpents and octopuses writhe, while a fat spider lords it menacingly over a town; here a causeway seems to lead nowhere, there a breakwater rears in panic, while ships are broken, dismasted and adrift. The sun fails to shine on landscapes whose mood is encapsulated by the title of one: Twilight, stubborn, black, hideous. These bleak visions become the bleaker for their oppressive lack of any human presence. The boats are unmanned, the cities unpeopled, the lighthouse lacks a lighthouseman, the pastures are absent of peasants and even animals. There are only three human figures – two of them hanged men; there is a stone angel and a statue of the crucified Christ. That manchineel tree casts a shadow in the shape of a human skull, while grotesque faces loom from cliffs and from the stalk of the vast threatening fantasy of Mushroom. In The Dream a sleeper’s tortured hand reaches out fearfully to ward off a floating, indecipherable face half-hidden by clouds.

You have to wrench yourself back into remembering that ordinarily Hugo’s world view was progressive, humanistic and cheerful. He looked forward to a time of universal fraternity. ‘In the 20th century,’ he once prophesied, ‘war will be dead; the scaffold will be dead, animosity will be dead, royalty will be dead; but Man will live.’ Which scores precisely nought out of four on what George H.W. Bush famously called ‘the vision thing’.