What It Costs to Pay Attention