Place the potatoes and 2 teaspoons salt in a large pot, add enough cold water to cover, and bring to a boil. Reduce heat and simmer until tender, 20 to 25 minutes.
Meanwhile, combine the oil, vinegar, mayonnaise, mustard, 1 teaspoon salt, and ½ teaspoon pepper in a large bowl.
Drain the potatoes and immediately toss with the dressing. Let sit, tossing occasionally, until the dressing is absorbed, 8 to 10 minutes. Add the celery and dill and toss to combine.