The Walker's were very heavy, almost twice the weight of the 1851's. The 1851 was chambered for the .36 caliber with ballistics similar to the modern .380. The recoil would be very light in a large pistol.
Hickok carried twin 1851's and fired out all the rounds daily and reloaded to avoid the danger of misfires.
Colt only made 1,100 Walkers, but 273,000 Colt Navys.
In the movies, you can see them being used by The Man with No Name and The Outlaw Josey Wales. There was a lot of fast gunwork in those.