In 1976, Kenneth Appel and Wolfgang Haken announced a proof of the long-standing Four Color Conjecture, which asserts that any "map" may be colored using at most four colors. It was a big deal, attracting attention from news media around the world (and that doesn't happen in mathematics very often). Given that the problem had resisted proof for more than 100 years you would think that mathematicians would have been more excited. The problem, you see, was that the proof relied on a massive calculation performed on a supercomputer at the University of Illinois rather than a written argument connecting statements with logic. In short, the proof was not beautiful.
Mathematicians love to use words like elegant to describe particularly efficient and aesthetically pleasing arguments. There is even a whole collection of such things called Proofs from the Book, which includes those proofs that are so perfectly constructed it is as if they were plucked from some Platonic firmament. The first example in the Book: Euclid's proof of the infinitude of the set of prime numbers, which most mathematicians know by heart. What makes it "beautiful?" It is simple, short, and has no extraneous ideas. We remember it so easily because it is precisely the most direct and obvious way to proceed.
Or consider The Method of Archimedes, in which we can compute the volumes of geometric objects by slicing them up and imagining the cross-sections sliding on an axis balancing on a fulcrum. In this way, using purely geometrical arguments (the most beautiful kind) Archimedes showed that the volume of a sphere is 2/3 the volume of the circumscribed cylinder, a fact he wanted engraved on his tombstone.
While most people imagine mathematicians doing arithmetic all day, except with really big numbers, the truth is that the discipline requires a remarkable amount of creativity and visual thinking. It is equal parts art and science. The late Maryam Mirzakhani, the first woman to win the Fields Medal, spent hours "doodling" on enormous sheets of paper as she worked out her mathematical ideas. These formed the basis of her elegant arguments in geometry and topology.
So what makes mathematics aesthetic? I would argue that in the end it comes down to three things: (a) a clear idea, (b) efficiently explained, with (c) a compelling visual to motivate. While we are often forced to rely on a grungy calculation to get the job done (the Four Color Theorem, or, more recently, Thomas Hales's proof of the Kepler Conjecture on sphere packings), mathematicians pine for the right diagram or geometrical insight to make us gasp with delight.