Olivia Rodrigo Surprises Fans at Billy Joel's Madison Square Garden Concert: 'Biggest Honor Ever'

Olivia Rodrigo and Billy Joel are one dynamic duo!

The "Drivers License" singer, 19, surprised attendees at the "Piano Man" crooner's concert at Madison Square Garden in New York City on Wednesday evening with a two-song performance.

Dressed in a black top with a brown and yellow patterned skirt and black leather boots, Rodrigo first performed her popular single "Deja Vu" — a track which references both Joel, 73, and his megahit "Uptown Girl."

The pair then performed that 1983 Joel song together, which appeared on his ninth studio album, An Innocent Man. They later embraced in a warm hug after the set.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

In a post on her Instagram Story, Rodrigo reacted to performing with Joel by sharing a backstage selfie of the musical pair.

"Biggest honor ever," she captioned the shot. "Still crying thank u Billy!!!!!!!"

In one other post shared on her Story, Rodrigo took a video of the crowd at the New York City concert venue singing along to Joel's classic tune "Piano Man."

For more on Olivia Rodrigo and Billy Joel performing together, listen below to our daily podcast PEOPLE Every Day.

"I SANG WITH BILLY JOEL AT MADISON SQUARE GARDEN TODAY ARE U KIDDING," the songstress wrote in white lettering on the post.

Joel also celebrated getting to perform with Rodrigo in a set of posts shared on his Instagram page as well.

In one post, Joel shared a professional shot of himself and Rodrigo backstage. The musician smiled next to the young pop star in a green suit jacket, which he accessorized with a black dress shirt and pants.

"Olivia Rodrigo makes a special appearance at Billy Joel's 82nd residency show at MSG!" the caption of the post read.

In another post, Joel shared a photograph of himself and Rodrigo rocking out on stage together. "good 4 u if you made it to @thegarden tonight! @oliviarodrigo hit the stage as Billy Joel's surprise guest 🎤💥," he wrote.

Joel's record-breaking residency at MSG began in January 2014, with the musician playing one show a month at the famed venue "as long as the demand continues." It's the first (and only) arena-sized residency in music history.

Since it began, every single show has been sold out. His 85th show in the residency is currently scheduled for Nov. 23, 2022. It'll be Joel's 131th lifetime show at The Garden.

Celebrity guests have been known to make cameos over the years, performing with Joel as Rodrigo did. Notable appearances have included Bruce Springsteen, Jimmy Fallon, John Mayer, Tony Bennett, Miley Cyrus, Bryan Adams, Paul Simon, Aerosmith's Joe Perry, John Mellencamp, John Fogerty and his own daughter, Alexa Ray.

As a Long Island native, Joel's music runs through the veins of those raised in the TriState area. Songs like "Big Shot," "Movin' Out," "Only the Good Die Young," "Scenes from an Italian Restaurant," "Still Rock and Roll to Me," "Uptown Girl," "New York State of Mind," "We Didn't Start the Fire, and of course, "Piano Man" typically have the crowd on their feet all night long, many singing along with Joel to every lyric.

They're just some of the 33 Top 40 hits Joel had as a recording artist, all of which he wrote and three of which landed at No. 1 on the Billboard charts. The 23-time Grammy winner has sold over 150 million records worldwide, including 1985's Greatest Hits Vol. 1 & 2 — which falls behind Michael Jackson's Thriller and The Eagles Greatest Hits as the best-selling album in history.

Among his many accolades, Joel's a member of the Rock and Roll Hall of Fame, a Kennedy Center honoree, and a winner of The Library of Congress Gershwin Prize for Popular Song.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.

Back in July 2018, Joel recalled his first show ever at MSG, which was on Dec. 14, 1978. "That was a milestone," he remembered in an interview with Newsday. "If someone would have projected that I would do 100 shows there, I would have laughed at them. I'd say, 'You're being ridiculous.' "

"The audiences are great," he added. "The venue is great. It's a world-class venue. To have a residency there is a dream already. I never imagined that anyone could have a residency at The Garden. We're a franchise. We're like a sports team. The whole thing has just been one crazy, exhilarating night after another."