Britney Spears for X Factor

There are several women's names currently being tossed around as potential judges for the upcoming season of "X Factor." Avril Lavigne, Janet Jackson and Britney Spears are the leading rumored contenders to replace Nicole Scherzinger and Paula Abdul for the show's completely revamped second season, which kicks off later this year.

In a recent press call, Simon Cowell would not confirm who producers were talking to, but he did not deny that Jackson was on the table. "You can probably count on two girls," fellow judge L.A. Reid added. "That we can confirm."

Cowell recently addressed the Britney buzz on Billy Bush's radio show, continuing to play coy about the chatter. "Well, I'm fascinated by her. The fact that she's still one of the most talked-about — not just pop stars, but people — in the world today means that you've got this star power. This girl's got resilience, and the fact that she's survived this and she's still hot, she's still having hit records and she's still controversial ... there's a reason for that.

"I would think that if she's managed to sustain a family, a career, which is tough, this is a walk in the park," he added, noting that he believes she would be a good judge. While Cowell seems to be dancing around the issue, British "X Factor" judge Louis Walsh may have let the cat out of the bag in an interview on U.K. talk show "Craig Doyle Live." He shared that Spears was indeed talking to producers about possibly joining the show.

"I know he is talking to Britney. I know he's definitely talking to Britney to do the American show. Absolutely true and maybe Janet Jackson," he said. In the end, Cowell summed up the current spate of rumors on that conference call. "There's a load of speculation, some true, some not true. It's true to say that a lot more people have entered the frame this year. We were waiting to see who was going to contact us before we actually contacted people," he said last week. "We're in that place right now, which is a good place to be in. We will meet a number of people over the next few weeks, mainly to explain to them that this is a big commitment when you do a show like this because of the mentoring aspect."