Preheat the oven to 425°F (220°C).
Spread the potato cubes on a large rimmed baking sheet and toss with olive oil, salt, and pepper.
Roast the potatoes for 30 to 35 minutes, flipping halfway through, until the edges are golden brown and crispy.
In a large mixing bowl, whisk together the mayonnaise, mustard, apple cider vinegar, dill, red onion, and celery.
Remove potatoes from the oven and let them cool for approximately 5 minutes to maintain crispness.
Gently fold the warm potatoes into the dressing until thoroughly coated.
Serve immediately to ensure the texture of the potatoes remains crispy.