With a recent update to the pinctrl-single driver and the fact
that the da850evm has both DM and OF_CONTROL working in both SPL
and U-Boot, some of the manual pinmuxing can be setup to only
be activated when either the driver doesn't have DM for it, or
when CONFIG_PINMUX isn't available (only during SPL).  If the
code ever shrinks enough to support PINCTRL in SPL, a lot of this
can go away.  This also remove some manual pinmuxing not needed
by SPL to give SPL a little more breathing room.

Signed-off-by: Adam Ford <aford173@gmail.com>