I think that the fact that he's letting you pick him up and hold him might suggest that he just needs to get used to it, as he's not been handled for so long.
fingers crossed, just keep going with him, the bites are a fact of life really, Mandi has a couple that she can't handle though.