I have some gaffers tape somewhere, I'll try to dig that up and cover the back of the fascia board to see if that helps.
I moved the camera forward by hand, and even with it right up against the fascia, the gutter causes glare.
Here's a daytime view. The wall would be an ideal spot for it, but I don't want it there for aesthetics.
Do you think painting the area the IR is hitting with flat white paint instead of the semi-gloss that's there would make a difference?