The original study that the age point came from just stopped tracking after that age. Which means that the conclusion of mid twenties being the magic age is a flawed bit of information.
What is true is that the parts of the brain that govern self inhibition and impulse control reach the norm for adults around the mid twenties. But it's developing long before that, and is going to be increasingly reliable from the mid to late teens in normative development.
However, that does mean that the norms for those factors are lower the younger the person is. This is not the same as incompetent, as you've rightly said. But it is reason enough to limit the kind of decisions a person can make using age as a rough guideline. That's why contracts with minors have extra rules, age of consent laws exist, there's drinking ages, driving ages, etc.