The Gunners took the lead through Olivier Giroud but Villa roared back, with Christian Benteke bagging a brace before Laurent Koscielny was sent off for the hosts in the second period.
Antonio Luna made sure of the Villans' three points in the final 10 minutes, as the home fans chanted, not for the first time, "Spend some f*****g money".
Rickie Lambert's week got better as he struck a late penalty to earn Southampton a 1-0 win at West Brom while Fulham triumphed by the same scoreline at Sunderland, courtesy of Switzerland international Pajtim Kasami.
Liverpool won the early kick-off, with Daniel Sturridge hitting the only goal in their victory at home to Mark Hughes' Stoke. The result should have been more emphatic for the Reds, who had goalkeeper Simon Mignolet to thank when the Belgian saved Jonathan Walters' late penalty on his debut.
Everton's Roberto Martínez era began entertainingly as they drew 2-2 at Norwich City, who led through Steven Whittaker. Ross Barkley's belting 25-yarder and Seamus Coleman's close-range finish put the Toffees ahead but Canaries debutant Ricky van Wolfswinkel equalised.
And at Upton Park Joe Cole and Kevin Nolan were on target for West Ham, who defeated Premier League newcomers Cardiff City.