Washington's summer secret is Dumbarton Oaks gardens

For visitors to Washington, it's the secret garden. Like ''The Secret Garden'' in Mary Hodgson Burnett's children's classic, it's sealed away behind a towering wall.

You have to know the flowers are there at Dumbarton Oaks, that behind the 12 -foot red-brick walls there are 10 acres of formal gardens, everything from Chinese wisteria to flowering jasmine and an oasis of roses.

Although the Dumbarton Oaks estate is known historically as the site of the 1944 international conference that gave birth to the United Nations, a visitor finds that other memories cling:

Cherry Hill in the spring, for instance, when the fallen cherry blossoms lie in thick pink drifts across the brown tree roots and early grass. Look up at Cherry Hill against the gray spring sky and the thousands of pink cherry blossoms give the air a silver-mauve light.

In spring on nearby Forsythia Hill, dozens of weeping forsythia bushes burst into a yellow blaze of flowers that cover the hillside like a three-alarm fire. It is so quiet then that you can hear the leathery green magnolia leaves fall with a gentle ''thunk'' on the rim of the pebble garden. Now, with summer, the crowds begin and the rose garden on a warm Sunday afternoon looks like a Jackson & Perkins rose growers' convention.

Over 100 rosebushes are in bloom, their colors ranging from crimson to fuchsia to salmon pink, seashell, lemon, garnet, cream, and the blond pink of the peace rose. Blue larkspur blooms in the gardens, along with purple and yellow pansies, pots of white jasmine, star of Bethlehem, honeysuckle, peonies, and perennials. The day lilies, gardenias, fuchsias, and oleanders are midsummer flowers; the chrysanthemums riot in the fall.

The family that left Dumbarton Oaks in trust to Harvard University in 1940 wanted to ensure that their secret garden would be shared with the public. Mr. and Mrs. Robert Woods Bliss bought the 19th-century Georgetown mansion in 1920. It was named not only for its towering oak trees but also for the rock of Dumbarton in Scotland; the grounds originally were part of a 1702 land grant owned by a Scottish farmer from Dumbarton-on-the-Clyde.

In 1922, Mildred Bliss and Beatrix Farrand, a celebrated landscape architect, began transforming it. They turned the former farm with cow paths and barns into the formal gardens which would vie with the traditional beauty of the English, Italian, and French gardens Mrs. Bliss loved. ''The Garden Twins,'' as these friends dubbed each other, devoted 25 years to perfecting the green and flowering vistas around the mansion. Like painters adding careful brushstrokes, they paid careful attention to every detail: to the hundreds of perfect pebbles from Mexican beaches for the pebble garden; the garden benches made of weathered teak from old British sailing ships; the Swedish wrought iron used throughout for decorations; the Star Garden fountain, which pictures Aquarius the water-bearer pouring falling stars and fresh water from a jug.

There is a harmony about the landscaped gardens and the mansion itself which mirror the Blisses' own attitude. They were extremely close in their tastes and outlooks, with a compatibility springing perhaps from their dual-family situation. The Blisses had been brought up together; as teen-age children her widowed mother had married his widowed father in 1894.

The Blisses were a handsome couple, both were independently wealthy, and they burnished their magnificent home and grounds when he retired. He had a 33-year Foreign Service career in Paris and Stockholm that was capped with the ambassadorship to Argentina. Instead of collecting souvenirs in their foreign travels, the Blisses collected art, beginning before World War I with pre-Columbian artifacts, and Byzantine art during the 1920s.

It was an international exposition of Byzantine art in Paris in 1931 that spurred the Blisses to begin acquiring only the best of this genre. Today, the Bliss Byzantine collection, housed in rooms of the mansion overlooking a mosaic pool, is considered one of the finest in the world.

On display there earlier this summer was a world-class exhibit of 12 Byzantine icons. The crown jewel of the exhibit was the 13th-century Macedonian painting of St. Peter, which Dumbarton Oaks has just acquired. Against a gilded tempera background we see a fierce-eyed, gray-bearded Peter, brawny looking as the fisherman he had been. His biblical keys hang like pendants from the neck of a gray-blue tunic covered with a greenish cloak.

Among the permanent collection are such treasures as the sixth-century Byzantine silver and gold book cover with a cross embedded on it, and a sapphire cameo bust of Christ Jesus, done in Constantinople in the 10th century.

The Blisses' pre-Columbian collection, which encompasses 50 years of acquisitions, is housed in a serene separate wing designed by architect Philip Johnson. Its round showcase rooms, gleaming with black marble, granite pillars, and glass walls, open onto a wooded view. An implacable looking statuette of an Aztec goddess named Tlazolteotl, with a child in a kangaroolike pouch, transfixes visitors at the entrance. It is known as the Mexican midwife goddess and is made of gray aplite speckled with garnets. It dates back to 1367. Among 600 wondrous items, the collection includes two Mexican monoliths, standing figures with almost snout-nosed animal faces, of grayish serpentine with red flecks, dating back to between 1100 and 500 BC, and Peruvian hammered-gold dance wands that look like smiling butterflies (600 BC-300 AD).

In addition to the collections, Dumbarton Oaks is also a renowned center for scholars in Byzantine studies, landscape architecture history, and pre-Columbian studies. Its research libraries covering those subjects include over 100,000 volumes. A separate Garden Library wing, built in 1963, was designed by Frederick Rhinelander King to house Mrs. Bliss's collection of books, prints, drawings, and manuscripts. It also includes paintings by Vuillard, Seurat, and Odilon Redon and works by Degas and Renoir. A rare El Greco, ''The Visitation,'' lights up a corner of the Music Room, a vast hall filled with antiques where public concerts are given. Igor Stravinsky once performed his Dumbarton Oaks Concerto here, commissioned for the Blisses' 30th wedding anniversary.

''Most people think we're a magnificent jewel box, and we are, but 75 percent of our activities go into advancing learning, lectures, seminars, etc.,'' says Dumbarton Oaks director Giles Constable. ''There is no break, to me, between that and our public message.'' The lure of Dumbarton Oaks, he suggests, is not any one specialty but its variety: ''It's wonderfully, incredibly varied. I'm sorry if anyone comes to see just one thing - if you come to see the gardens, drop into the collections; if you come to see the collections, stay for a concert . . . so that you get the feeling of interaction between the life of the mind and the life of the spirit with the arts.''

That attitude is exemplified in one of the current exhibitions here which meshes indoor art with outdoor art. It is a small but precious collection on medieval gardens. The exhibit includes ''Planting the Patrician Garden,'' the best-known medieval text on that subject, done in 1470 by a retired lawyer known as Petrus Cresentius in Flanders. A page is open to an illustration of such a patrician garden, with crenelated towers in the background and a medieval blue sky overhead. The page is rimmed with a flower and gilt border.

In another showcase is a 16th-century silver gilt statuette from Krakow of Jesus as a gardener holding a shovel and wearing a broad-brimmed hat. The work of art was inspired by gospel verses in John 20:14-17.

In a sense, Dumbarton Oaks is, as Smithsonian Magazine has suggested, like a medieval walled garden. It is a garden in which everyone can find a favorite niche, a treasured spot to return to: perhaps the great, gray-brown satsura tree from Japan, with low limbs like wooden sculpture you can perch on, or the long green sweep of the French steps behind the mansion.

For Mr. Constable there is a special place: ''As night comes on, to look through the ellipse over the valley and see the fireflies there is an incomparable sight, like a fairy Christmas tree. That's to me the most beautiful. In the Orient . . . they even give firefly parties,'' he says, sounding as if he'd like to give one himself at Dumbarton Oaks.