THWARTING TERRORISM; Controversial guidelines give CIA, FBI a freer rein

President Reagan's newly issued executive order on intelligence activities is designed as much as anything to help rebuild the morale and confidence of the US Central Intelligence Agency.

It is also designed to give the CIA and FBI an additional degree of freedom as they attempt to counter the threat of terrorism. In theory at least, under the new guidelines the intelligence agencies will be better able to cope with the type of assassination teams that Libya is believed to have trained to attack top US officials, including the President himself.

Reagan administration officials came to the conclusion that an earlier Carter administration order had been unduly restrictive and was impairing the performance of the intelligence agencies.

But most experts on the subject seem to agree that there is no simple solution available to solve many of the problems that have beset the CIA and other intelligence agencies. The failure of the intelligence agencies to foresee developments in Iran, for example, derived in part from a broader problem within the US government. The CIA's capability to gather information from Iran and to analyze it was limited. But no one at the top of the government seemed to be asking the right questions or be capable of facing the bad news when it came.

More than 100 organizations concerned with civil liberties have, meanwhile, condemned President Reagan's executive order as providing insufficient protection of Americans' constitutional rights. These groups are concerned that the new order allows the CIA too much freedom to spy on Americans both at home and abroad. Some of these groups are also concerned that the CIA may once again be getting into the business of helping to overthrow foreign governments.

Within the government the problem looks different. Some officials of the CIA itself are said to be concerned that the executive order goes too far in the direction of allowing spying on Americans. The 17-page order for the first time permits the CIA to collect ''significant'' foreign intelligence from Americans within the United States or abroad even if those Americans are not suspected of being agents of foreign powers. But the order also contains many of the provisions that had been included in the more restrictive order drawn up by the Carter administration. It bans assassination by the spy agencies and provides for continuing congressional oversight. It does not go as far toward easing restrictions on the agencies as some earlier drafts of the order were purported to have gone.

One former high-ranking intelligence officer said that the new executive order is likely to bolster the morale of many officials at the CIA because it makes clearer the ''dos'' and ''don'ts'' which apply to the agency, eases restrictions to a degree, and places the goals of the CIA in a more ''positive context'' than the Carter order did. ''The Carter order emphasized suspicion of the intelligence agencies,'' the former official said.

He said that many officials at the middle levels of the CIA were still in a state of ''shell shock'' from earlier criticisms of their activities and that the recent Senate questioning of CIA director William Casey's business dealings and appointments had not helped. Casey's deputy director for operations, Max Hugel, was forced to resign last July following allegations of financial misconduct.

Many of the experts on intelligence are scornful of the civil libertarians' concern that the CIA may now be suddenly ''unleashed'' to engage in ''covert action'' overseas. Such action is usually defined as secret activity aimed at influencing political conditions in other nations.

A symposium of some 60 former American and foreign intelligence officers, academics, and other experts agreed for the most part that the CIA's capability to engage in successful covert action had eroded to the point where it was incapable of responding effectively to the demands placed on it by the White House. Their findings have just been published by the Washington-based Consortium for the Study of Intelligence.

What is still uncertain is how far the Reagan White House will go in pushing for such controversial activities as covert action and surveillance of Americans. That will depend in part on implementing orders and classified guidelines which are not made available to the public.

What is clear is that the Reagan administration has decided to reemphasize the use of human spies against ''targets'' such as Libya in addition to the technological collection of intelligence. Technological collection - through means such as spy satellites - was heavily emphasized under the Carter administration.

In another development, Libya denied sending ''hit teams'' to the United States to assassinate President Reagan and said that such reports were a product of the CIA's ''fantasy farm.'' Libyan leader Muammar Qaddafi, in an NBC-TV interview, also strenuously denied the reports, and claimed that he is himself the target of a CIA assassination plot.

US officials have said that they take seriously reports that Qaddafi has sent teams of his own assassins to the US. But they have also said that the evidence is so far inconclusive. Americans with past links to Libya have been questioned by the FBI and Secret Service.