Bottling the nuclear genie: new chapter begins

US-Soviet arms talks will be handled with steely toughness gloved in velvet. New evidence shows that just as much toughness may be needed to keep other nations from going nuclear.

The United States and the Soviet Union are speaking politely to each other these days, a tactic which both sides find useful for the moment.

Much of the world applauds. But behind the screen of polite talk - a good deal of it designed for West European consumption - the two superpowers continue to pursue exceedingly tough-minded approaches to countering each other.

Neither side seems to feel any great sense of urgency about moving toward compromise positions which might lead to agreements on limiting either medium-range or long-range nuclear weapons. In particular, for the Geneva talks now getting under way the two sides have put forward seemingly attractive positions on medium-range missiles which in reality place them far apart.

All indications are that the Reagan administration still believes the only way to parley is first to arm. Administration officials are convinced that unless the US moves ahead with certain new weapons programs, the Soviets will have no incentive to bargain. Despite expressions of hope for the Geneva talks, many of these officials continue to view the Soviet Union as a power which understands brute force better than anything else.

On the Soviet side, mistrust of American intentions apparently still prevails. The Soviets give every sign of thinking that, through the European antinuclear movement and their own propaganda efforts, they can stop the deployment in West Europe of new American Pershing and cruise missiles.

Hence, despite the more conciliatory public posture assumed by both sides, what one expert calls a ''mini-cold war'' continues. And the Soviets continue to pursue their long-standing aim of splitting West Europe away from the United States.

From the American viewpoint, the problem is seen as one of countering the Soviet military buildup and what is described as Soviet expansionism - while not appearing to be so aggressive in doing this as to lose West European public support. Administration officials are convinced that in the end the security of the West will depend more on Western military strength and American security guarantees than on any arms control agreements.

In a speech which he was preparing for delivery in London on Nov. 30, director of the US Arms Control and Disarmament Agency Eugene V. Rostow is expected to argue that arms control agreements contain no magic in themselves and that a satisfactory outcome in negotiating with the Soviets is hardly guaranteed. In Rostow's view, it is not the lack of arms control agreements but a Soviet policy of expansion, fed by the growth of the Soviet armed forces, which has produced increasing instability in the world.

What is new in this situation is that President Reagan, out of deference to West European sensitivities, has agreed to go to arms control talks with the Soviets without first insisting that the Soviets moderate their ''expansionist'' behavior. But a State Department official warned that there was no guarantee that the President would continue to renounce, as he did implicitly in his Nov. 18 speech, the ''linkage'' between Soviet behavior and continued arms control talks.

The State Department official said that some American experts thought, for example, that Nicaragua was on the verge of obtaining new Soviet fighter planes. Should that happen, it could easily shatter the public politeness now prevailing between Washington and Moscow.

Another source said that State Department officials have argued, meanwhile, for a degree of flexibility over the long haul in the negotiations with the Soviets which open this week in Geneva. The State Department is said to have developed a ''fallback position'' which would carry the negotiations beyond the proposals set forth by President Reagan in his Nov. 18 speech. The President offered to cancel the planned deployment of 572 American new Pershing II and cruise missiles if the Soviets would ''remove'' their new SS-20 and older SS-4 and SS-5 missiles which are already in place and targeted against Western Europe.

The fallback position would include sharp reductions in - rather than the elimination of - the Soviet SS-20 missiles in return for equally sharp reductions in the planned deployment of new American missiles in Europe. It might also allow for certain types of American fighter aircraft to be included in the Geneva negotiations if the Soviets insisted, but only if Soviet aircraft of comparable range were also included. Top Reagan administration officials have apparently not yet agreed, however, to the fallback position or positions.

But Dimitri K. Simes, a Johns Hopkins University expert on the Soviet Union, thinks the Soviets have little incentive to make concessions which the Reagan administration would find acceptable because they are still not sure that the administration means what it says when it talks about building up American defenses.

Simes, who describes current US-Soviet relations as a ''mini-cold war,'' says , for example, that President Reagan's recent interim decision to deploy up to 100 MX missiles in heavily armored ''silos'' rather than going for the more ambitious mobile MX missile project was probably seen by the Soviets as a sign of political weakness rather than strength.

''The Soviets have no doubt about Reagan's belligerency toward them,'' said Simes. ''But their assessment of his belligerency and their assessment of his effectiveness are two different things.''