College or self-study? Is a good school important?

* I am 20 years old.

* I live in a developing country.

* I've spent three years in a crappy college (private).

* I have a good job as a programmer at a nice startup.

* I have the chance to leave my actual college and enroll at the best university in my country (It is free and SO MUCH HARDER).

* I wish I had a better background in mathematics and algebra. I'm awful :(

* I actually love and enjoy programming.

My heart says to leave the crappy college and enroll in a better school (although I will struggle a lot with the difficulty) but my brain says 'go finish your studies and bore yourself to death doing UML stuff in the easy school'.

Any thoughts?