Fiction from three 'world class' writers; Mailer's excesses sink Egyptian extravaganza; Ancient Evenings, by Norman Mailer. Boston: Little, Brown & Co. 709 pp. $19.95.

What happens when a two-time Pulitzer Prize-winner spends a decade on his ''big'' novel, and it turns out so solemnly obscene as to be ridiculous? For one thing, it divides the heavy-hitting professors who are enlisted to review it.

''It is, speaking bluntly, a disaster,'' writes Amherst's Benjamin DeMott of Norman Mailer's 700 pages on death and reincarnation among the pharaohs (the New York Times).

''A new and permanent contribution to the possibilities of fiction and our communal efforts at self-discovery,'' concludes Columbia's George Stade (The New Republic).

''More than before, Mailer's fantasies, now brutal and unpleasant, catch the precise accents of psychic realities within and between us,'' says Yale's Harold Bloom (The New York Review of Books).

These reviews, like a sampling of nonprofessorial ones, stress the new novel's relentless preoccupation with perverse views of sexuality that Mailer has dealt with to some extent before. The verdicts break down roughly into: failure despite some displays of expected talent; success despite all the shortco ings; and somewhere on the fence.

To be sure, there are effective scenes; a pharaoh in battle, for example, orating on top of a cage with a roaring lion in it. And Mailer has developed a kind of antique, Arabian Nights language to go with his ancient evenings, when a man who has lived four lives tells more about pagan corruption than anybody with one life to live wants to know. You can believe it when Mailer says he has tried to make sure there is not a single Judeo-Christian idea in the book. Like William Golding's ''The Inheritors,'' which imagined rudimental mental processes for prehistoric man, Mailer imagines thoughts and passions as alien as myth on the walls of Egyptian tombs, with charnel-house imagery to match. In terms of technique he gives himself no more quarter than the reader by devising narratives within narratives, characters within characters, mentalities within mentalities, that do hang together if you pay attention, class.

Yet for all the effort the result is negligible in relation to the spectacle of a major author's art and judgment regressing so far as to treat old-fashioned barracks-room porn as part of a profound historical discovery. It is beyond parody to the point of drowning out what Mailer evidently wants to say about violence and magic, wealth and waste, power and humiliation.

In previous books, whatever their welh-known excesses, Mailer has often conveyed the earnestness of someone wanting to be candid no matter what the cost to himself or his targets. He speaks as if he is utterly serious about ''Ancient Evenings'' (and two planned sequels!), too. But does he really lack what once was called the mother wit to see how ludicrous the effect is? Or is the old champ, to use his favorite boxing metaphor, chuckling in his corner, ''If I can get them to believe this I can get them to believe anything''?