On March 18, when the Metropolitan Museum of Art opens an annex at Madison Avenue and 75th Street in Manhattan, it will be attempting to shrug off the ghost of a museum past.

The specter is the Whitney Museum of American Art, which called the iconic Marcel Breuer building on that corner home for nearly five decades. In an eight-year deal, the Met is leasing the Breuer building from the Whitney— which relocated to its dazzling new Renzo Piano–designed home last year—in an attempt to expand its Modern and contemporary-art presence.

It’s not a simple real-estate leap, because of how inextricably linked Breuer’s granite-clad inverted pyramid was with the Whitney’s identity. “It is, along with the Guggenheim, a truly bespoke museum building that was commissioned to house a changing series of exhibitions, and, to some degree, a collection,” says Sheena Wagstaff, the Met’s chairwoman of Modern and contemporary art, who is overseeing an international contemporary program at what will now be called the Met Breuer.

Founded in 1914, the Whitney bounced from home to home until its trustees decided, in 1963, that it should have a permanent one. Breuer, the Hungarian-born, Bauhaus-trained émigré, was at the height of his career. “It was that moment when New York became the center of the art world in the ’50s and ’60s,” says Wagstaff. “There was a concerted effort on the part of the artists represented in the Whitney to create a truly American idiom.”

Breuer and the Whitney were riding the same wave of post-war American triumphalism. But Breuer’s building was misunderstood as brutalist—uninviting, bunker-like, and out of context with the neighborhood’s brownstones— though he never allied himself with the movement. Part of the Met’s goal is to continue to chip away at those misreadings and celebrate Breuer’s egalitarian approach to engaging visitors. “We want to cleave to the architect’s original intentions,” says Wagstaff, and remove the accretions of decades to accommodate changing technology and bigger crowds.

On an early February tour, while workers buzzed about, Wagstaff gushed: “I cannot emphasize enough how bloody amazing this building is.” Yes, she is British, and the former chief curator of the Tate Modern. “There is this brilliant tension between the sensuality and colors of the materials he uses and the way he deals with the ceilings and the austerity of what the building is meant to represent.” She points to Breuer’s bush-hammered concrete walls in the lobby, which are framed by smooth boardformed edges. “I mean, look! He frames the concrete. It’s kind of a fetishist’s delight.”

Wagstaff and the Met worked closely with preservation architects Beyer Blinder Belle (BBB)—the Met’s master planner since 2013—to study Breuer’s life and the building, and then determine what to restore and how. They declined to disclose the project cost. “Breuer called the materials in this building materials of the earth. And he championed and specified the fact that natural materials age beautifully. They take on a patina,” says John Beyer, a BBB founding partner. “A challenge for any architect dealing with a masterpiece is knowing what not to do as much as what to do.”

On the museum’s five gallery floors, Beyer and his team simply vacuumed Breuer’s famous precast-concrete coffered ceiling. They carefully restored granite and plaster walls where needed. It was Breuer’s cleft-bluestone floors that took the most beating over the years, and the architects patched them in certain places, replacing only about six stones in the entire building, says Beyer. They finished the floor with a wax that is neither shiny nor dusty-looking.

The lobby received the most attention. Here Beyer’s team removed all evidence of the retail shop that the Whitney had inserted. This left holes in the bush-hammered walls that competed with the round indents from the original snap ties, so they were selectively patched. The sculptural overhead light array was repaired, and each bulb was replaced with an LED, giving all the bulbs the same value. The architects also created a new ticketing desk in walnut, which, in color, matches Breuer’s teak stair railings. Breuer’s original concrete and granite “book bar” at the back of the lobby remains in place and will be the only retail element in the space. Behind the book bar, a new 6-foot-tall, 23-foot-long LED media wall (in a black as matte as cloth) displays exhibition and museum information. This aligns with another of the Met’s big goals—to establish the Met presence.

Astonishingly, Breuer’s structure is not landmarked, but it is a contributing building to the Upper East Side Historic District and it is protected. The Met and the BBB architects want to dispel the erroneous notion that Breuer meant for the building to be a fortress. He also didn’t mean for its street-fronting “sunken garden” to be perceived as a moat.

In order to emphasize that this open courtyard underneath the building’s cantilevered entry portal was meant to be an urban retreat, the architects removed unsightly dangling cables—part of systems that included now-defunct cameras—that hung down into the below-grade space. Nodding to Breuer’s plans for a green space to soften the area between the sidewalk and the museum door, they commissioned landscape architect Günther Vogt to plant a row of Quaking Aspens that will grow to varied heights and be uplit at night. This lighting will, says Beyer, be a dramatic sight from the double-height glazing that extends from the sunken garden up to the ceiling of the lobby.

The Met Breuer will open with a show of the contemporary Indian artist Nasreem Mohamedi’s drawings and photography (which Wagstaff organized to align with the waffle ceiling, since Mohamedi plays with and subverts the grid in her work), as well as a 500-year survey of unfinished pieces by artists from Renaissance masters to Lygia Clark. A restaurant will open in July on the lower level where the Whitney had a café. Beyer says that working on the building has been a highlight of his career—and a lesson in careful handwork. “We are thrilled that the Met had the vision to see the way to do it right,” he says.