如何专门为两个输入分配功能

I am having an issue I cannot figure out regarding assigning a specific function to an input. I created a color picker with two different color options (inputs). Each one adds color to a different div. In what I have created, I do not understand how I can control which input assigns the color to the outside or inside div. As you can see in my fiddle, I have the inner color input first and that changes the outside color and vise versa for the other div.I am having an issue I cannot figure out regard