A number of women have accusedR. Kellyof domestic abuse and sexual assault throughout his career —and now, the people who grew up closest to him are speaking out.

In Lifetime’s new six-part docuseriesSurviving R. Kelly, the 52-year-old R&B singer’s brothers — Bruce and Carey — break their silence opening up about the man accused of abusing women for decades.

Eight women who allege being abused — or who tried to help family members — open up in this week’s issue of PEOPLE. Representatives for R. Kelly replied “no comment” to PEOPLE’s request for a response to allegations made inSurviving R. Kellyand our interviews.

Carey Kelly.

Carey 1

“Robert is not cut from a cloth of being street,” Carey, who is R. Kelly’s younger brother, said. “He didn’t come outside, he stayed in the house and played with his keyboard.”

Bruce, who is R. Kelly’s older brother and is interviewed from Chicago’s Cook County Jail as he is currently incarcerated, also said “Rob was shy and very timid. If you stared at him in his face he would cry.”

“I said, ‘Robert what’s wrong with you man?’ Bruce said. “He just broke down again, ‘I’m just tired of all these kids teasing me.'”

Bruce Kelly.

Bruce 1

Once the family moved out of the projects, though, Carey said “a lot of things changed for the worst.”

Earl Gibson III/Getty

r-kelly-brother

“It teaches you to definitely be sexual earlier than you should have, than you’re supposed to,” he said. “You know, no different than putting a loaded gun in a kid’s hand — he gonna grow up being a shooter, probably. I think it affects you tremendously when that happens at an early age. To be more hornier. Your hormones are up more than they would normally be. Mine was.”

Carey said he believed R. Kelly when he said he was abused because he claimed it also happened to him.

Carey 2

Despite their difficult childhood, R. Kelly’s brothers said he was able to find an escape through music.

“He became obsessed with doing music because that was his outlet, it took his mind away from reality, which was pretty rough,” he continued.

R. Kelly.Gilbert Carrasquillo/FilmMagic

r-kelly-3

But still, Bruce said R. Kelly was the same shy kid he always was. “We had to urge him on to sing; he would sing with his back turned,” he said.

After R. Kelly appeared on Natalie Cole’s talent showBig Breakin 1989, Carey said that’s when his career really started taking off. Not long after, he was signed to Jive Records.

While out on tour with R. Kelly in the past, Carey claimed that his brother “had some guys, who, if he said, ‘Go get some girls from the mall,’ they’d go get some girls.”

During his interview, Bruce also said that R. Kelly liked “younger women.”

“You have people who have fantasies about different things,” he said. “I like older women, go figure, you know. But that’s just a preference. Everyone has preferences. So what is the big deal? What is the big issue with my brother?”

When R. Kelly married Andrea Kelly — who is also featured in the Lifetime series — in 1996, Bruce said he was “shocked.”

“It just looked to me that they weren’t compatible,” he said. “She just seemed different than what he was accustomed to having.”

Bruce-2

Andrea and R. Kelly were married until 2009 and have three children together. They met two years before they wed when Andrea auditioned as a backup dancer for the star. She would go on to handle most of the choreography for his tours, videos and live performances.

They weathered marital ups and downs during their time together. In 2005, Andrea obtained an emergency protective order against the singer after he allegedly physically abused her — she then dropped the order weeks later. The couple then filed for divorce in 2006 but were still trying to make things work.

Earl Gibson III/Getty; Paras Griffin/WireImage

r-kelly-andrea

“Me and my brothers, we all have our ways with our girls,” Bruce said. “We’re very jealous guys. We don’t want our girls doing certain things, [you’re] not gon’ see our girls talking too much to people, that’s just the way we are.”

“We used to talk with Andrea and the other dancers and stuff and have fun,” he continued. “But after they got married, there was no more of that. It was like she wouldn’t even speak to us. It was hard to know if we could speak to her. Robert [chartered] a plane with a banner that said, ‘Andrea, will you marry me?’ And then a dude dropped down in a parachute with a ring and she said, ‘Yeah.’ So it was a secret ceremony. He was really sneaky and I would have loved his brothers to be there with him, for memories if nothing else. Fame is one thing, but when you start to lose certain life’s precious moments because of fame, I think I’d do without it. I felt bad, I could see that she wanted her people there, her mom there, her dad there. But that’s Robert though.”

He concluded: “After the marriage she got real quiet. She was really quiet, and I think that’s how Robert wanted it.”

andrea-kelly

In 2002, R. Kelly was indicted after a video surfaced showing a man engaged in sex acts with a woman who some witnesses testified was 14 at the time of the recording. Both R. Kelly and the 14-year-old denied it was them, and R. Kelly was never charged with assault. In 2008, R. Kelly was found not guilty after being indicted with 21 counts of child pornography.

A report fromBuzzFeed Newsreleased in July 2017 alleged that R. Kelly haskept at least six womenin his Chicago and Georgia properties who allegedly fulfill his desires and are punished if they break any of his “rules.” These allegations have never led to formal charges.

For more powerful stories from alleged victims of R. Kelly, pick up this week’s issue of PEOPLE on newsstands now.

Surviving R. Kellyairs on Lifetime at 9 p.m. ET from Thursday, Jan. 3 to Saturday, Jan. 5.

If you or someone you know think they are being abused, call theNational Domestic Violence Hotlineat 1-800-799-7233 (SAFE) or 1-800-787-3224 (TTY) now for anonymous, confidential help, available 24/7.

source: people.com