Yahoo scoring is not exactly intuitive, but in this case, the TD does count against the D. While looking for something else, I came across this FAQ in Yahoo Fantasy Help:
Why is the Points Allowed category for a defense different than the final score?
In Yahoo! Sports Fantasy Football, scoring for the Points Allowed category in leagues using Team Defenses will include the following situations:
All TDs scored against the defense.
All FGs scored by the opponent.
All PATs (Extra Points and 2-PT conversions).
Special Teams Scoring (kickoff and punt returns for touchdowns, as well as blocked punts returned for Touchdowns).
All other scoring categories chosen within the league.
** Please note that points scored against the team's offensive unit (i.e., interception and fumble returns for TDs, FG blocks returned for Touchdowns, and safeties) will NOT be counted in the Points Allowed category.