Sounds like a subtle (and normal) approach. What you can do is to have several development branches, for each function or error correction you are working on. This allows you to switch between branches depending on what you want to work for today. But if you are a lone developer in your project, this may just be redundant.
Just the last update of my answer, I found this article and it looks like a very good approach to version control using git: http://nvie.com/posts/a-successful-git-branching-model/ And by the way, this is very close to how we use version control at work.
However, this is probably most relevant for projects with several people.