“It was so confining,” Madison, 36, tells PEOPLE exclusively. The author of a second memoir, The Vegas Diaries, moved out of the mansion in 2008 after seven years living there with then boyfriend Hugh Hefner.”It was a place where you’re made to feel like as a woman, your value decreases as you age,” she says. “And it decreases if you have something to say. It’s really not healthy.”
Her female roommates, (six when she first moved in then dwindled down to three), didn’t help.
PHOTOGRAPHS BY GREGG SEGAL
“When I arrived, I saw women tell lies about other women and get them kicked out. I got scared really fast. They were always watching every move I made. There were sharks in the water.”
For exclusive photos and much more from Holly, pick up the new issue of PEOPLE, on newsstands Friday
Madison grew more uneasy with each passing year. “I never felt comfortable in a situation where by the time women were 28 they were supposed to be put out to pasture,” she says. “And it was a culture that really encouraged women who acted stupid. It was a constant struggle.”
Continues Madison: “I realized what a drama queen [Hefner] is and how much he loves drama. I really started seeing his true colors and I couldn’t make excuses anymore. I had to get out of there.”
After releasing her first book, Down the Rabbit Hole: Curious Adventures and Cautionary Tales of a Former Playboy Bunny, last year, Hefner responded to Madison’s allegations about him and about life at the mansion.
“Over the course of my life I’ve had more than my fair share of romantic relationships with wonderful women,” Hefner, 89, said in a statement to PEOPLE at the time. “Many moved on to live happy, healthy and productive lives and I’m pleased to say remain dear friends today. Sadly, there are a few who have chosen to rewrite history in an attempt to stay in the spotlight.”
“I guess, as the old saying goes: You can’t win ’em all,” concluded Hefner’s statement.
“I can look back at all the missteps that seemed so tragic at the time,” says Madison, “and realize I learned from them. It all comes together.”