The controls act on the selection. With an arrow selected, or the layers it connects, a change shows on the canvas at once. With nothing selected, the same controls only set the look of the next arrow. If a pink bar at the top says “Editing style”, you are changing a saved style, and only arrows that use it change; press Done to leave.
Try it on your own file
Free, with unlimited arrows, no account. Or draw a few arrows in the browser first.