Looking at it from a fan of Batman or The Avengers is where you get the it wrong. Batman in many ways can beat the Avengers one-to-one. Yet, when together, it becomes more difficult.
Batman would definitely win but only in a few circumstances.
When They Are Partying.
And they love to party.
But this brings me to the next place
The capes home and blood. Yes if the Avengers were in Gotham city Batman's home court and element of surprise would be too much for them.
When He Has A Heads Up.
Having the connections as he does and the Avengers big mouth to the media, the bat , or even Bruce could know of their arrival.
How does it go down?
Number 1, take out hulk, it could be a knock-out dart on Banner or some type of distraction to hold him up or lead him away. Then Thor, most likely by an nullifing agent then smashing his head onto his own hammer causing a knockout. Batman (who wrote his own curriculum for high school at age 12) would most definitely hack, if it wasn't the first thing he did, Tony Stark's suit along with the communications of the team. Or he could use his own iron suit ( which he already has) to out fight Tony. Then using his League of Shadows training, will take out H-eye & Black Widow. Then just straight out fight Cap and (though tough) out right, out maneuver him.
And don't for a second think that he can't. He is the only Justice League member who Superman trusts can beat him, the only one who created, memorized, and used the list on how to defeat the entire Justice League, by himself, the Avengers, no problem