Can I put a Semi-Perm Color on top of Perm Colored Hair?

I have permanant color-treated hair. My natural color is a medium brown and right now the color of my hair is several shades lighter to a golden blonde color (didn't bleach my hair, just used lightest blonde permanent color I could find and 20 volume developer). Now I'm seriously thinking of going darker, but just to a very light warm brown color. Not wanting to screw things up too much, I was thinking of using a semi-permanent color that will wash out, (in case I don't like the color), on top of the permanent color I have now. Can I do this with good results and without messing up the texture of my hair, etc., or is there another way I should go on this? Thanks so much for any help!

Capelli