PALO ALTO, Calif.—Here in the center of Silicon Valley’s tech boom, one of America’s wealthiest enclaves is wrestling with an uncomfortable dilemma: whether it can afford to lose the city’s only trailer park.

A block from multimillion-dollar homes and a few miles from the headquarters of Google Inc. and Facebook Inc. sits Buena Vista Mobile Home Park, with 117 units that are home to about 400 residents, many of them Hispanic laborers.

Erika Escalante, 29 years old, has lived in the park much of her life. The mobile home she owns with her husband is watermelon red with white trim. The yellow trailer three doors down, where she lived during high school, is still home to her parents and youngest brother. Her sister’s family has a green unit in the park, where some rusty trailers still sport Christmas decorations in summer.

Soon, they may be forced out. With property values soaring, the park’s longtime owners, 44-year-old Joe Jisser and his parents, are fielding inquiries from developers eager for a rare large slice of Palo Alto. The value of the park’s 4.5 acres could be as much as $55 million, local real-estate agents say.