Issue with jQuery :not selector

Issue with jQuery :not selector

I have a search icon that when hovered over reveals an input text search
box. I'm using jQuery to add a class to the input to reveal the text box
when the search icon is hovered over. That is working fine.
The next function is that if the user decides not to click in the search
box and elsewhere on the page, the search box goes away. That's working
fine too.
The only remaining issue is that if the user clicks in the search box, it
also goes away which I don't want.
My HTML:
<input type="text" class="form-text text-hidden" title="Enter the terms
you wish to search for." value="" size="15" id="edit-search-block-form-1"
name="search_block_form" maxlength="128">
<input type="image"
src="http://d3j5vwomefv46c.cloudfront.net/photos/large/802431483.png?1377197793"
class="form-submit search-hover" alt="Search" id="edit-submit">
My jQuery:
$(".search-hover").hover(function () {
$('.text-hidden').addClass('hover');
});
// Now remove the class in case of a false start or the user decides to
do something else.
$("body:not(.text-hidden)").click(function(){
//alert('The body click is working!!!');
$('.form-text.text-hidden').removeClass('hover');
});
Note that I'm attempting to use the :not selector to indicate to only
remove the hover class if clicked anywhere but the search box but it's not
working.
I've got a fiddle here