It's a time constraint dude.
You've different basic types of legs (ex: Humanoid, Chicken Walkers).
If people actually wanted this game released within the decade we had to make some exceptions due to having limited modelers and animators. This means that the one or two guys we had available to animate (while they were still doing other shit) had to rig and create animation sets for every leg type, then adjust those leg animations to accommodate the particular mech they were settling on all while making sure they blended correctly.
Upon release, we wanted to put as many mechs in as possible so we set aside into groups every mech that could share legs with other mechs of the same leg type. Then we tasked our animators with setting up the correct rigs, constraints and animation sets then blended them together and got more mechs in by doing this.
Also, lets take a look at those concept sketches you posted. I'd like you to emphasize, from a modeling and animation standpoint, in technical terms, how you'd go about actually modeling and animating those monstrosities with their organic curves and inadequate pivots and joints. One problem I, as a modeler have, is fighting with shitty TRO images where the artist goes "LOL LOOKS COOL TO ME BUT CAN'T REALLY MOVE WHO CARES LOLOL". Some designs are so fapping retarded there's no way physically possible to have them actually rotate along a pivot enough to animate a full range of motion without clipping through geometry and looking like ass just to stay with some lazy douches crappy source art.
Our mechs have multiple pivot points for a full range of rotations based around actually moving. Hip joint, knee join, ankle joint and 3-bone foot for the possibility of minute animations and physics (we intended toes to lift up, hang, make contact with the ground) and so forth. Now, try doing all this for almost 30 different mechs.
So, the reason you see so many similar legs is due to manpower and time constraints. If more people were willing to slave for free for years on end, we'd be more than happy to have different legs, animations and models for everything.