The term designer has and always contained the visual aspect as well. Sure you can understand and mimic the role of the designer, but if you do not possess any of the artistic abilities of a designer, can you really call yourself a designer without having to fake your skills somewhere down the line? It's like a cook who can mix, chop, cut, slice, but hasn't got any sense of taste or better yet, has no trained presentational skills so everything looks like baby food.
Sure, you can be an artist who borrows or steals, but can you be a true artist?