Reinventing the wheel

[xlucas]

Not sure if this should go here or in "Programming". It really is about programming, but it's a philosophical topic, so... that's why.

Since I started programming, about 25 years ago, I have been criticised a few times for one particular thing and I've seen other being criticised for that same thing. It's the typical "Why are you trying to reinvent the wheel?" or "There's no point in doing that, when there is this other thing". I've been thinking hard about this through the years and now I'm feeling like I can write about it.

I've realised I have a tendency to reinvent the wheel... and I've realised I'm one of many. I don't do this because I don't understand the tools that are available or because I dislike other people's work. I do this because I feel that the part that is fun about programming is the creative aspect. To me, if a tool already exists, that doesn't make it useless that I create something similar. Of course, my work may be unnecessary for others, but it does a certain good service to me and I think that many programmers can benefit from reinventing the wheel. It will make them feel better and it will be good practice. It will make them learn. I will make them aware of how something works internally when they decide to use it as a tool.

For example, some time ago, I wrote a small lib that reads and saves TGA files. I'm sure there are tons of tools to do just that and also, there are formats more convenient that TGA, but making my own has been great and I like using my own code because I know how it works. If my program has a bug, I will know if the bug is in the new code or in the library because I wrote it. If the library is 3rd party and it's buggy, I will be wondering what I'm doing wrong in my module-level code for years until I realise it's the library. But also, now that I've written my lib, I know what it's been like and I am more likely to detect a bug in a library that's not mine if it's the same type of library. This is just an example. There are infinitely many cases like this.

Now that I've made the point clear, I would like to ask you guys a few things:
- Have you ever been criticised this way?
- Do you personally feel a natural aversion to people who try to reinvent the wheel? A desire to strangle them or something? :P
- Would you recommend others to learn by reinventing the wheel?
- What is your insight about the new programming trend that's more about putting pieces together than about creating the pieces themselves? Do you feel like a piece-joiner or a piece-maker?
- Do you think this is a matter that only hobbyist should consider while paid programmers should ignore?
- Anything you'd like to add?

Thank you all for reading this probably very unusual post. I'll be glad to read your comments, even if you criticise me... again :)

[D.J.Peters]

I hope this 58 seconds give you an idea what I think about "do what you want to do !"

youtube: 58 seconds from Stan Lee

Joshy

[Julcar]

Not sure if this should go here or in "Programming". It really is about programming, but it's a philosophical topic, so... that's why.

Since I started programming, about 25 years ago, I have been criticised a few times for one particular thing and I've seen other being criticised for that same thing. It's the typical "Why are you trying to reinvent the wheel?" or "There's no point in doing that, when there is this other thing". I've been thinking hard about this through the years and now I'm feeling like I can write about it.

I've realised I have a tendency to reinvent the wheel... and I've realised I'm one of many. I don't do this because I don't understand the tools that are available or because I dislike other people's work. I do this because I feel that the part that is fun about programming is the creative aspect. To me, if a tool already exists, that doesn't make it useless that I create something similar. Of course, my work may be unnecessary for others, but it does a certain good service to me and I think that many programmers can benefit from reinventing the wheel. It will make them feel better and it will be good practice. It will make them learn. I will make them aware of how something works internally when they decide to use it as a tool.

For example, some time ago, I wrote a small lib that reads and saves TGA files. I'm sure there are tons of tools to do just that and also, there are formats more convenient that TGA, but making my own has been great and I like using my own code because I know how it works. If my program has a bug, I will know if the bug is in the new code or in the library because I wrote it. If the library is 3rd party and it's buggy, I will be wondering what I'm doing wrong in my module-level code for years until I realise it's the library. But also, now that I've written my lib, I know what it's been like and I am more likely to detect a bug in a library that's not mine if it's the same type of library. This is just an example. There are infinitely many cases like this.

Now that I've made the point clear, I would like to ask you guys a few things:
- Have you ever been criticised this way?
- Do you personally feel a natural aversion to people who try to reinvent the wheel? A desire to strangle them or something? :P
- Would you recommend others to learn by reinventing the wheel?
- What is your insight about the new programming trend that's more about putting pieces together than about creating the pieces themselves? Do you feel like a piece-joiner or a piece-maker?
- Do you think this is a matter that only hobbyist should consider while paid programmers should ignore?
- Anything you'd like to add?

Thank you all for reading this probably very unusual post. I'll be glad to read your comments, even if you criticise me... again :)

Sometimes I have the same thoughs, especially when I have to alter other's work and I don't have enough time to look how it works, people when contracting programmers thinks we know everything of everyone's code, and that's why I preffer to write my own code so I can look where it fails, or where can be improved. In the case there is a time consuming task, I will look around for code written as close as my coding style, in fact I've been amazed many times when I find code that looks just if it was written by me.

[marcov]

Now that I've made the point clear, I would like to ask you guys a few things:
- Have you ever been criticised this way?

All the time. It doesn't help that I'm on a development platform (Delphi) that is notorious for its very conservative one-man shops.

- Do you personally feel a natural aversion to people who try to reinvent the wheel? A desire to strangle them or something? :P

It depends on the reasons. There should be good reasons and a goal for doing so. Such control in directions that are your core business(as in: the main thing(s) your apps do) are IMHO much more forgivable.

I e.g. also have load/save routines for bmp and png, but (speedy) mass-storing of images is one of the core aspects of our applications. (camera based inspection systems)

Sometimes the aspects are speed, sometimes it is simply the available stuff is not suitable, and if I need to heavily modify it, I do it based on my own terms (read: first I port it to our development platform), rather than start maintaining C#/Java/Python whatever development systems.

If a component is usable (usually C++ code), I wrap the C++ in a DLL and use it from Delphi/Lazarus (*).

But we also have some auxiliary apps, e.g. for own administrative purposes and some relating to us making our own electronics. (most notably I'm working on automating export from kicad projects to our pick-n-place machine). These are much more using externally made components.

- Would you recommend others to learn by reinventing the wheel?

Again only in topic core business, otherwise it just detracts. Using something existing might get you quicker to the point that you get to spend time on the really important stuff.

Note that this is only a guideline.

- What is your insight about the new programming trend that's more about putting pieces together than about creating the pieces themselves? Do you feel like a piece-joiner or a piece-maker?

I've done both, but lean towards maker. Even when I was more in a piece joiner position (making backend webapps, mostly in Delphi but later moving to C#/ASP.NET), I still did a lot of the "hard" parts like framework, and other atypical work. Even if I did business code work, it was usually the rather odd jobs.

That job also had a slightly larger team (3 programmers including me, and a IT savy manager), and that makes reusing components slightly easier, because every person has its own preferences, but also knowledge areas, or would specialize on some of the stuff.

Being able to divide up the maintenance work for components made reusing easier than when I'm alone (like now, when I'm the only coder in the company)

- Do you think this is a matter that only hobbyist should consider while paid programmers should ignore?

It depends on the hobbyist. If you are e.g. work on an app that is distributed to large amounts of people, you have fairly clear goals and e.g. reducing dependencies might make deployment easier. But that is usually really targeted, not out of some random not-invented-here syndrome.

If it is just for yourself however you risk spreading yourself too thin and get stuck in sideways related topics too easily, never finishing anything.

(*) our main apps are Delphi, much of the internal apps and the like are in Lazarus. A handful of main apps do both, usually because the customer wants the source to be handed over, and we deliver them as a zip with project sources AND a ready to run lazarus development environment).

[badidea]

I think that the name of the topic is not right, it is not about 'reinventing the wheel', but about 'making your own wheel'. Now today's wheels are a bit complicated, so lets replace 'wheel' by 'chair'. Why woud one make his/her own (wooden) chair instead of buying one from Ikea?

[dodicat]

IKEA sources it's wood from over 50 countries.
Sweden has vast areas of coniferous forest but of course they don't want to be seen de-foresting Sweden so they de-forest other countries instead.
We really have to be careful what we do these days, not going absolute vegan or extreme things like that, but supporting sustainable home industries as best as you can and avoid using big powerful corporations.
Ships wheels were made of teak or monogamy, I do like teak but monogamy I am not a fan of.
I notice that waste plastic makes really great furniture, quite often it is manufactured in your own country.
Regarding using other peoples algorithms, I only hobby code so I try to make up my own methods.
Maybe if I coded for a living I would use the best stuff around to get a job done.
Sorry to be such a bore regarding wood, but human activity must change or there will only be a few generations left in us.
It is a while since I have been in Holland, do the barges still have those huge wooden wheels, or have they small electric tillers these days?
Have a peek badidea, next time you are down by the canal.

[badidea]

My intention was not to turn this topic into a discussion about trees and wood, but about doing things yourself even if this takes much longer, might be more expensive and/or lower quality.

Back on trees: I have been in Sweden (~10 years ago). To me Sweden looked like 70% trees, 25% water and the remaining 5% I could not see due to all the trees. Making chairs from trees is not so bad. Chairs store CO2. And after ten years, you throw away the chair, burn it (which does release the carbon into CO2) but the energy can be used. Much better then burning trees directly for energy. Also after 10 years, you can have a little new tree grown back. I can imagine that the people in Sweden don't want all their trees converted into chairs for the whole world. In the Nederlands there are no trees left; all converted to wooden shoes, barges and windmills. I live I the southern part of the Netherlands now where there are less canals.

[Imortis]

For me, I like to make things from scratch to learn how they work. I don't do that for my job, but when I am working on things that are not time sensitive, it is quite nice to learn the ins and outs of something but doing it from scratch.

[jj2007]

I frequently reinvent the wheel. Today I wasted several hours trying to find a shapefile editor that allows me to crop a map. I have a map of Europe that includes Russia, but I don't want Russia dwarf the rest of Europe, so I want to limit the map at xx degrees East. Sounds simple, right? It takes one minute in PaintShop (drag a rectangle to select the area, press Shift R, done), but unfortunately I need the map vectorised, and PaintShop doesn't know what a shapefile is.

No luck, I installed 400MB of QGis, an utterly bloated QT-based crap that can do everything except cropping a map. Tried another one called "EveryMap", completely useless. OpenJump looks OK, it's "only" 50MB, but it cannot crop a map.

So what I will do is sit down and write the cropping routine myself. That implies studying the map format and going Assembly but it will be sheer fun ;-)

P.S. Disinstalling QGis took over 2 minutes, but over 2GB more disk space was worth the effort, lol

[marcov]

jj2007: yeah, the openstreet GIS stuff is particularly hairy.

BadIdea: quite ironic, considering that "Holland"'s etymologically comes from Holt-Land, iow wood-land.

[jj2007]

jj2007: yeah, the openstreet GIS stuff is particularly hairy.

It is, absolutely. But in the meantime I found a way to crop a map with OpenJump. This implied finding first a way to convince OJ to present itself in English instead of Italian, because the documentation on the web is, obviously, mostly in English. Then you need a new layer called "Fence" (only this name works), and afterwards it's simple. Well, kind of - still clumsy like hell, but nonetheless I decided not to reinvent that wheel today ;-)

[BasicCoder2]

When my brother in law asked me what I was programming and I explained it to him what I was programming his comment was, what is the point in that? For me it was the same point as playing golf or tennis (his social activity). You can buy a chair or build a chair really doesn't matter. I also combine programming and electronics and I guess there is no point in that as well except it is cool doing it myself.

However productivity on complex projects does require building on the work of others not spending time reinventing it unless you can do it better. Nature works that way. All complex systems are made up of working modules built up over time. Of course it may mean using less than perfect modules. I remember the memory limitation on the old pc and all the "work arounds" to access more memory as it became available.

[xlucas]

Hey, guys! I have to say I'm positively surprised on how much I've been understood. I guess I never commented much on this topic and assumed I was pretty much alone in this train of thought, but I was wrong. Thank you all very much for your comments. It's nice to realise we here have more in common than I thought.

While each of us can have an idividual way of thinking and doing the things, there clearly is a global trend at each point in time. I'd say in the 90s and early 2000s, programming was a more individual or closed-group-oriented task while nowadays, they concept of an open-community-oriented project is a lot more common. My point is that regardless of what the trend is at any time, I think that always, back then and right now, both approaches are good and have their reasons and it's great that one can find an environment to work with people who see it the same way.

I reckon some projects make more sense if community oriented, like is the case of FreeBasic, since it's not just a project, but also the tool we use for other projects. But sometimes, I'm working on something and I feel I need to share it some other way. I need the project to be just my own and to communicate with other people who are also working on their own projects. Inspiration and help don't have necessarily to come in the form of directly having somebody else contribute to your code. If you always work as a community, then non of your works will be personal and sometimes, you just want them to have your colour and smell. And if you built your own chair, you can not only sit on it, but you can tell your visitors "Hey, this chair, you know, I made it!" and it feels good.

Off-topic
My grain of sand on the topic of wood and ecology... I think wood is a great natural material that goes back to nature. The problem is not that we use wood. The problem is that we are too many. Every single ecological or economic problem and many social problems as well could be solved by just controlling how many people we are. This includes accumulation of trash, global warming, pollution, poverty, etc. If you become a vegetarian, you will cease consuming meat, but if you decide not to have children, there will be perhaps two fewer people in the history of the world that will ever eat meat. Just like you guys, I do not want to go off topic, but because this is an important thing, I decided to comment on it. If you guys want and think the topic deserves its own space, we could start one. I think it's an important matter. Cheers

[jevans4949]

Throughout history people have been literally reinventing the wheel. If you think about modern car wheels, they are vastly different from prehistoric cartwheels. I seem to remember seeing a TV documentary which mentioned how cartwheels had improved in efficency over the period of the Roman Empire and the Dark Ages.

[BasicCoder2]

That is more an improvement on the wheel rather than a reinvention. Over time inventions like the wheel are improved upon or modified for different uses. Our technology and science progresses by using the understanding of those that have gone before. Over time knowledge accumulates. Some times it is same problem resulting in a similar solution when people share the same knowledge base. Like Leibniz and Newton with calculus. Or Darwin and Wallace with natural selection. In nature knowledge accumulates in the dna.