It's funny how the negative elements of your past can skew the present positives and I yhink that's exactly what Alec is experiencing. He hated the town, he hated his mom and how she rained her bullsh*t down on ecery aspect of his life as a teenager, but the beacons of light -- Finn, Milo, River, that bunch -- float on the surface of his mind while the dark elements eventually sink to the bottom (to be forgotten).
I hope his subconscious self keeps leading him to Finn. The little girl is Milo's and, even if she isn;t, the fact that she is Finn's and he loves her will be enough to temper Alec's feelings. Again, he may hate the town, but someone like Finn will weather distance, will weather time. Finn is, like, the definition of "absence will make the heart grow fonder," Alec.