Ask HN: How did you start your own project?

Without much rigorous CS and software eng background, I am always curious how projects are started. What exactly did you do at the beginning? Did you start with something dirty and then kept refactoring it or you already had the whole framework in mind already (at least a simple one)?