Deep trees are flexible enough to fit almost any data (low bias) but are extremely sensitive to the specific training sample (high variance). Change a few data points and you get a completely different tree. This is exactly why Random Forest (bagging) works so well with trees: it keeps the low bias while averaging away the high variance across 500+ independent trees.

Without replacement, you'd have to choose a subset size: too small means each model sees too little data, too large means all models see nearly the same data (low diversity). Bootstrap elegantly solves this: each sample has N points (full learning capacity) but ~37% are missing (natural validation set) and some points appear multiple times (adds randomness). It's the Goldilocks solution for creating diverse samples of full size.

Boosting reduces BIAS by building up complexity gradually. If each base learner is already complex (low bias), boosting has nothing useful to correct — it just memorizes noise. Deep trees as base learners in boosting leads to severe overfitting because each tree overfits to the residuals (which are increasingly noisy as you correct more). That's why XGBoost uses max_depth=3-6 (weak learners), not max_depth=None (strong learners).

This is THE core insight: Bagging averages strong, independent, high-variance models to smooth out their noise → variance reduction. Boosting combines weak, sequential, high-bias models that each fix what the previous ones got wrong → bias reduction. Both produce ensembles better than individual models, but they work in opposite directions on the bias-variance spectrum.