Fort damaged after '40 golf balls hit walls'