When I haven't seen her face, so sometimes he imagines her when being intimate with another. And then he leaves and ends up missing her and even later in the song he says, And I have rarely held another And during these intimates time she did seem younger, her leathery hands(whether from work or age) turned to a velvety touch. And he learned much because he was younger, and she was experienced but she never took advantage of him, later in the song it even says that we had the need to feel the hunger, so he was obviously agreeing to this and he wanted to. And he can so easily notice how she wants to feel something like that again, so they did it.
This song is literally about how an older teen, went to work for a woman on a ranch or something and one day he can see the obvious longing in her eyes, because he husband is gone, so she's alone.