Whitney Houston and Bobby Brown: Inside Their Volatile Relationship

While the world first learned about Whitney Houston‘s drug addiction when she first went into rehab in 2004, some blamed her husband, rapper Bobby Brown, at first.

But Brown, who s written a new memoir Every Little Step, denies he was the first to introduce her to drugs and claims in an ABC interview with Robin Roberts (airing in full tonight) that he first saw her doing drugs on their wedding day.

“People want to blame Bobby, [but] there are things that predate Bobby,” a close family friend told PEOPLE after her death in 2012. “Her and Bobby coming together just happened to be one of those things – like people are drawn together. They had this sort of Bonnie and Clyde mentality where it’s them against the world and them having to defend themselves against the world and that bonded them even tighter.”

RELATED VIDEO: Bobby Brown Says He and Whitney Houston Failed Bobbi Kristina

“People think Bobby Brown introduced her to drugs – that’s not true. If anything, she introduced Bobby to real drugs,” says a music source. “Bobby pretty much just smoked weed. They had a lot more in common than people realize … the rough-and-tumble guy from the projects in Boston with this bad boy image and America’s princess. She loved him. She truly did. In some ways, Bobby provided stability in her life, and she had someone who truly cared about her. Unfortunately, it became too much for both of them to handle. They blew through their money, they started doing more and more drugs together and they were both dealing with sinking careers.”

Meanwhile, she made him one of her advisors when his career took a free-fall. “She did a lot of things to accommodate him,” says the close family friend.

The relationship also turned volatile when claims surfaced that Brown allegedly hit Houston. In 2003, he reportedly was charged with misdemeanor battery related to alleged domestic violence. But when asked about how far fights between the couple went, Houston told Redbook in 1999: “I do the hitting.”

Finally, in 2006, Houston filed for divorce and was granted full custody of their daughter, Bobbi Kristina, in 2007.

Despite their difficult relationship, Brown and Houston “had finally gotten to a place where there was no drama at all” when she died tragically in 2012, a close relative told PEOPLE exclusively. “”He loved her as the mother of his child and as a woman who he had spent his youth with.”

But for her mom, Cissy Houston, it was already too late. As she told PEOPLE in 2013 after writing her own memoir Remembering Whitney: “She finally got rid of him, but it was too late. I blame him for the way he treated her. And he didn’t help [with her drug issues]. Damn sure he didn t do that. He was no help to her at all.”

As for the stories then swirling around her daughter’s life, which have continued in the years after her death, Cissy claims “people that want to remember her badly will do that, and the people that really loved her, will remember her the way she was.”

•With reporting by TIFFANY MCGEE and STEVE HELLING