You're right. There is an awful lot unsaid. There are now 5 books altogether, and I'm still deciding what to do with them. Very tidied up versions of I and II are now published in ebook and I think I'll be looking to getting book III and IV out this year. I'm not certain about V because that's set 10 years in the future and is a definite end, and I'm not sure if I'm ready to let go of Silver yet.
With regard to Silver and Asher, I think it's something that had to be. The relationship with Silver and River was unequal from the start. Silver (as far as he can remember) has never had a relationship. The one with David doesn't count because it was doomed from the start and could never have been a natural one. To a great extent Silver was totally reliant on River, and I don't think they could ever have stood a chance of having an equal relationship until he had been with someone else. Put simply Silver needed Asher in order to find SIlver and until he found Silver he could never have a proper relationship with River, if that makes sense.
Please remember that Silver is not a normal person and cannot react in the same way a normal person would. He has lived the whole of his life that he can remember as a conditioned slave and he has no experience of independent living. He relied on River for everything and is literally cut adrift without him. Silver has no experience of normal life or even of making decisions for himself. He has no frame of reference and no chance of understanding the situation with River and Ben. River, is caught in a difficult situation, literally stuck between a rock (Ben) and a hard place (Silver) He naturally feels a strong protectiveness about Ben because he's his brother and because of what they've been through BUT what he's refusing to acknowledge is that Silver actually needs him more. Ben, even though hes much younger than Silver has a much better grasp of the situation and is happy staying with Sam, while Silver is not. In my opinion River was cruel to Silver - unintentionally so, but cruel nonetheless. Silver had to lave or he would have been irreparably damaged because River could never have given him what Ariel and Asher did. Freedom in a way that he could understand.
I hope that makes things a little clearer.