To understand the Science behind this phenomena we need to look at the difference between sidereal days and solar days.
If you're floating in space and watching the Earth rotate, the time taken for the Earth to complete one rotation about its axis is called the sidereal day. It actually takes only 23 h 56 min to complete one rotation, a bit shorter than what we would consider a day.
Take a look at the picture above. Starting at noon on Day One with the Sun directly over your head, at the end of one sidereal day, the Earth has completed one rotation. But because the Earth has shifted just a little bit, it's not noon yet where you are: the Sun is still a little bit to the East (the north pole is coming out of the screen). It takes some extra amount of time to rotate you back to noon position. This extra amount of time is about 4 minutes long.
Now, if the Earth moved in a perfect circle, then every day, this extra time needed to rotate you back to noon position will be the same, every single day. But because the Earth's orbit is ever so slightly an ellipse, the Earth's speed changes as in moves around the Sun. When the Earth is somewhere near its closest to the Sun (this is called perihelion), the Earth moves faster than usual, meaning that it'll take more time to shift back to noon position. This means that at 12 pm on your watch, the Sun is still a little bit to the East. During this time, the Sun rises later and later as the difference between 12 pm on your watch and actual noon becomes larger and larger. Perihelion occurs in January, which is why the fajr timings get later. Likewise, when the Earth is somewhere near the furthest point from the Sun (aphelion), the Earth moves slower than usual, and the Sun starts shifting to the West and rising earlier, bringing the prayer times forward.
In fact, if you were to take a picture of the Sun at sunset or sunrise every day at the same clock time, you'll see the Sun tracing out something like a figure-eight shape called the analemma
This curve is a result of a combination of the axial tilt (23.5 degrees) of the Earth shifting the Sun north and south, and the elliptical orbit effect that I've just described, which shifts the Sun up and down. Prayer times will be adjusted accordingly.
