One dimension of modernist poetry is introducing entities in surprising contexts, such as wheelbarrow in Bob Dylan’s feel like falling in love with the first woman I meet/ putting her in a wheelbarrow. This paper considers the problem of teaching a neural language model to select poetic entities, based on local context windows. We do so by fine-tuning and evaluating language models on the poetry of American modernists, both on seen and unseen poets, and across a range of experimental designs. We also compare the performance of our poetic language model to human, professional poets. Our main finding is that, perhaps surprisingly, modernist poetry differs most from ordinary language when entities are concrete, like wheelbarrow, and while our fine-tuning strategy successfully adapts to poetic language in general, outperforming professional poets, the biggest error reduction is observed with concrete entities.