Not without replacing the bracket, but since it's a very simple bracket, there should be no reason at all you can't. At a guess, I'd say that you could probably even fit a 5.25" speaker in that location, but for anything larger than a 4" I'd really want some sort of enclosure.
The factory bracket is just a bent piece of metal. There's a hole for the speaker, with screw holes, then it comes down and bolts to the chassis in 2 locations with 6mm bolts. Practically zero baffling. I believe that the "crossover" is simply that the 3.5" speaker can't produce any bass, so it doesn't (but it tries.)
The main reason that I wouldn't put larger speakers there is that it's a crappy location. (So's the rear deck, kind of a toss-up as to which is worse.) Each one points right at the back of the front seats ahead of them, and it skews the staging something awful. Oh yeah, and to make enclosures for it is annoying to pull those interior panels 3 or 4 times in order to fab something up.