Connor,
I've only just started wearing knee pads, they take a lot to get used to.
Elbow pads I've always warn and never had a problem with.
If it's the injury side your concerned about then i think that the knee pads would offer more for you. As the impact on your knees tend to be more than your elbows, i.e. Sliding & kneeling.
Yet the team I play for, only 2 of us wear the knee pads and everybody wears elbow pads.
Knee pads can restrict your movement sometimes.so try and borrow a set first as They may not be for u.
Just my opinion
All the best