Refactoring is cool. Really cool. It also means something very specific. There's a book and everything. If you haven't read it, you should. (Hint: it's by Martin Fowler)
Refactoring means reducing code smells by making small, incremental, orderly, and very specific changes to existing code. Make such changes (one at a time) allows one to improve the structure of existing code without changing its functionality. Always a good thing. Refactorings have very specific names like "extract method" or "encapsulate field" or "push member up". Again, there's a book. You can look them up.
Where is this going?, you might ask. I already know this, you say. Cool. Then let's move on to what refactoring (Refactoring?) isn't.
Refactoring doesn't mean changing code because you think your way would have been better. It doesn't mean rewriting things from scratch because you have a different opinion. It doesn't mean starting over again and again in pursuit of the perfect solution to every coding problem.
Those other things have names (which I won't mention here for the sake of any children reading this), but "Refactoring" isn't among them. There's a tie-in here with another term we all love, "Agile". Refactoring fits into an "agile" process after you've made everything work they way it should (i.e. passes the tests) to make it easier to work with the code on the next story/backlog item/iteration. The point of agile development (IMHO) is to write as little code a possible to meet your requirements. It doesn't mean redoing things until you end up with the least possible amount of code, measured in lines. Again, that has a different name.
Sometimes code needs to be fixed. More often than we'd like, in fact. But if you are (re)writing code in pursuit of the most bestest, don't call it Refactoring. It confuses the n00bs.