Why does a nearby tree seem to move faster than a distant tree when an observer moves sideways?

The phenomenon you are describing is known as movement parallax (or motion parallax). It serves as a vital monocular cue for depth perception, allowing the brain to judge distance even when using only one eye.1

When an observer moves sideways, the retinal images of objects in the environment shift at different rates depending on their distance from the observer. Because a nearby tree is closer to the observer, its position relative to the observer's line of sight changes significantly with even a small lateral movement, causing it to appear to move across the field of vision more rapidly. Conversely, a distant tree is much further away, meaning the angular change in its position is minimal as the observer moves, making it appear to move much more slowly.1

This difference in the rate of displacement provides the brain with essential information about the relative depth of objects in space.1